How do we know the data is robust?

NIWA scientist Gregor Macara and MetService Meteorological Data Manager Kevin Alder explain how they collect standardised data, while Our atmosphere and climate 2020 authors Drew Bingham and Bonnie Farrant explain quality checks to ensure the data is accurate.

Discussion questions:

• What does Kevin mean when he uses the word ‘standard’?
• What are two ways the data is checked?
• Why is it important for climate data to be both robust and transparent?

Transcript

DREW BINGHAM

We collect all kinds of data on the climate here. The temperature and rainfall, the greenhouse gas concentration data and ozone and UV data that are collected in Lauder in the South Island.

GREGOR MACARA

Predominantly, it’s collected through climate stations that we have located right throughout the country. These climate stations have a number of different instruments. We have five or six hundred of these weather stations throughout New Zealand measuring things generally in real time 24 hours a day, 7 days a week.

It is important to ensure the ongoing accuracy of these stations once they’ve been established, and so we do that in a number of ways. One of those ways is a technician will go out into the field at least once per year to check all the instruments and make sure they’re operating as they should be. When these technicians set up these weather stations, they’re setting them up according to a set principle and various other rules and regulations according to best practice.

KEVIN ALDER

This is what we call a Stevenson screen. And it has a unique design, as you can see with these louvres, so the air flows through it without being impeded, and this is the standard way to measure air temperature. And you’ll also note it’s positioned about a metre and a half off the ground, and that again is the standard. And also, we try and measure it over grass, so it’s consistent wherever the measurement was made.

This is a rain gauge. In this case, we have two devices, so we’re ensuring data consistency by checking one against the other. Again, this is all about ensuring the data is to a particular standard and a certain quality.

GREGOR MACARA

We also have technicians in the office. And they’re monitoring the data that’s coming in, in real time, and there’s also not only manual checking of that data but also automatic checking through computer programs.

DREW BINGHAM

There’s a lot of different quality checks along the way. There will be data checks to see if there’s missing data, and when it’s brought over to the people who are going to use the data, they’ll also run checks on it to make sure that it matches up.

BONNIE FARRANT

We make sure that any numbers that we create for the reports are accurate and have been interpreted correctly for the nature of the data, because often there’s stuff you can and cannot say depending on how the data was collected and what the data looks like. So we are the quality gate for the data and all the figures in the report.

That’s my favourite part of the job is cleaning and investigating the data. I’m pretty much a detective, so we go through nearly every row in the dataset and make sure that it makes sense. One time, we had a value in 1 year that the change had been bigger than a whole century, so of course you go back to NIWA and ask, “Hey look – is this OK? Is this correct?”

We write programs that explore every single line and pull it apart in so many different ways. Coding is traceable. It’s almost like writing a recipe, and it allows anyone to kind of rerun the code on the dataset and get the same results.

It’s all about making our results robust. We need to be transparent and very accurate, so we often do rerun the programs multiple times.

And we do make our datasets public from the report. You can often download a dataset and repeat all our analysis and see what we’ve done, or you can play around with it and see the issues that are close to you.

Acknowledgements
Drew Bingham, Ministry for the Environment
Gregor Macara, NIWA
Kevin Adler, MetService
Bonnie Farrant, Stats NZ
Aerials of Lauder Atmospheric Station; ute driving to Baring Head; Principal Technician Gordon Brailsford analysing air samples; animation of weather stations across New Zealand, NIWA
Gregor Macara calibrating weather station and working with data, filmed by Lana Young, NIWA
Bonnie Farrant coding to check data, Stats NZ
Scroll of Stats NZ website, Stats NZ