Preserving World War II Naval Data

Have you ever wondered how scientists are able to determine past weather, water, and climate conditions? In the time before weather satellites (pre-1960) and standardized large-scale observation networks, monitoring was relatively limited both on land and in the ocean. Observation was also heavily dependent on United States. Naval vessels, which had a somewhat consistent presence on the seas as well as the necessary instrumentation.

Datasets for this pre-satellite era were created in the 1950s and 1960s, though significant uncertainties exist in instrumental marine weather datasets from the early 1900s (as noted by IPCC Assessment Reports five and six). The period from 1940–1945 in particular is likely affected by data artifacts (data flaws that can be caused by issues with instruments, techniques, or conditions) in United States. Navy collections.Due to the scarcity of observations for this period, biases and artifacts in these data can cause major problems in reconstructing this part of the weather, water, and climate record. 

Preserving the Data

The original Naval records for the first half of the 20th century were recently identified by anthropologist Kevin Wood, then with the University of Washington, who knew of the potential issues with these datasets. NCEI holds the surface marine (aerological) records whereas the National Archives facility in College Park, MD holds the Navy deck logs, which include meteorological observations and documentation of events on and around the ships. While these original records can be helpful for correcting issues in the datasets for this time period, they had never been digitized, preventing most scientists from accessing them and using them for research. Wood applied for a grant to image and digitize these records and was successful in securing funding for the project. 

The first goal was to scan the aerological records stewarded by NCEI, a collection of 158 boxes of weather observations taken on board US Navy ships from 1929–1961, with the bulk from 1930–1945. Student interns from the Atmospheric Sciences department at the University of North Carolina at Asheville (UNCA) were hired. Using equipment purchased under the grant, they completed that collection by imaging 60,001 pages of US Navy ship weather observations.

The team then moved on to a secondary goal. As the National Archives facility was not fully reopened due to the ongoing pandemic and copies of the United States. Navy deck logs for the World War II time period (1942–1945) in the form of microfiche reels were discovered in the NOAA, The National Oceanic and Atmospheric Administration NCEI archive, the team decided to image them. Of the 366 reels in this collection, 169 reels were scanned, resulting in 107,041 deck logs scanned, organized, and given a standardized file name. 

Dataset Renovation

The files scanned and cataloged under this project will be archived at NCEI and shared with a global team working to correct biases in sea surface temperature and other data collected by Navy ships during this era. Though final results remain to be seen, the potential impact of bias correction as well as the identification and removal of artifacts from these Naval datasets created in the 1950s and 1960s can be significant.