health & science

Old weather records could predict snow in Auckland

Machines will transcribe handwritten weather records from as far back as 1860 to help predict future weather patterns 

On the subantarctic Campbell Island, a worker in charge of the daily recording of weather observations started his job with excellent penmanship. 

After three years living on a remote island where rain falls 325 days of the year and wind gusts reach 96 kilometres per hour, his penmanship devolved into “chicken scratches”.

His weather observations and accompanying comments are soon to be deciphered in a project aiming to use artificial intelligence to transcribe the handwriting used in weather records from up to 150 years ago. 

The information he and hundreds of others recorded at various weather stations around New Zealand could give insights into future weather. 

NIWA principal scientist Dr Andrew Lorrey explains: 

“Old weather data and old scientific data are really useful from a perspective of getting as much in-depth knowledge as far back in time as we can go ... Having those data in databases or detailed maps is useful for us to contextualise contemporary events and set up expectations for what the impacts from weather might be in the future.”

Globally, records prior to World War II aren’t widely digitised.

The challenge has been how to get boxes of sometimes fragile paper-based records, some written with quill and ink, into a form where the data can be used. Entering each entry by hand is one option, but with thousands of records, it’s a hard, slow slog.

“We're talking about hundreds, you know, hundreds of boxes, you know, dozens and dozens, if not hundreds of stations.”

Each sheet of paper has 31 rows with 25 columns for different weather observations such as temperature, barometric pressure and rainfall. 

Penmanship ranges from the “incredibly clear to chicken scratches”.

“It is something that we are going to need a machine to help us with if we want to accelerate the capture of this old knowledge. We think we’ve got a really good plan.”

The machine power is coming via Microsoft’s AI for Earth project which aims to support people working on environmental challenges.

Microsoft New Zealand’s Senior Cloud and AI Business Group Lead Padi Quesnel said it’s the scheme’s second grant awarded to a New Zealand project.

The plan uses the help of citizen scientists who have already been working on transcribing the documents to train machines. 

“Citizen scientists are already typing up small sections of those archived pages … when that's been verified and uploaded by other citizen scientists, anywhere up to 10 times, they classify that as being 100 percent accurate.”

This is the starting base to train the artificial intelligence model to automatically recognise letters, numbers and decimal points and capture the data from the scanned or photographed pages.

Machines will transcribe written comments like this one from the day in 1939 where snow fell over much of New Zealand. Photo: Supplied

“An example, we're using the 1939 weeks of snow everywhere in New Zealand as a reference point to compare to an incident that occurred not quite as drastically, but similar in 2011, where we had a lot of snow in obscure places like Clevedon in 2011, and comparing the two.”

Patterns leading up to the events could help predict whether Auckland’s Mount Eden is about to be dusted with five centimetres of snow again, as it was in 1939. 

Quesnel doesn’t think this has been done before for weather data and said two other global weather organisations are also interested in solution. 

He calls it a “cool problem” he became interested in. While the project of scanning pages was underway, he felt just storing the scans wasn’t tapping into the full potential. 

“They still weren't really maybe harnessing old data to make new data … to be able to make an impact on society and New Zealand is something that got me really interested in it.”

NIWA’s Lorrey said the results will form part of a global effort being conducted by climate scientists around the world as part of the ACRE project, who are reconstructing the weather on Earth.

“It's really important for us, because it helps to clarify things like extreme weather events and things like extreme climate patterns ... That for us is really important for planning and developing infrastructure that's going to be more resilient to extremes, especially in a future climate that's changing, we're expecting more of that to happen.”

We value fearless, independent journalism. We hope you do too.

Newsroom has repeatedly broken big, important national news stories and established a platform for quality journalism on issues ranging from climate change, sexual harassment and bullying through to science, foreign affairs, women’s sports and politics.

But we need your support to continue, whether it is great, small, ongoing or a one-off donation. If you believe in high quality journalism being available for all please click to become a Newsroom supporter.

Become a Supporter

Comments

Newsroom does not allow comments directly on this website. We invite all readers who wish to discuss a story or leave a comment to visit us on Twitter or Facebook. We also welcome your news tips and feedback via email: contact@newsroom.co.nz. Thank you.

With thanks to our partners