Census red flags raised in March
Problems with low census response rates were flagged at Stats NZ daily operational meetings in March, David Williams reports.
In the days leading up to March 6, census day, Stats NZ was hit with a raft of unexpected problems.
Internal documents, released to Newsroom under the Official Information Act, show that on March 2 it had been discovered that 4263 building consent addresses were missing from the address list. Another 1000 private dwellings, it was found, had disappeared from the system. On the same day, it was asked “how to enable” foreign embassies, which were initially not recognised. A list of 139 embassies was sourced a few days later.
In IT problems, age and sex data had been removed from the census “response store” because of Deloitte security recommendations, a problem that couldn’t be fixed until after census day.
Stats NZ’s summary of daily operational meetings – released to Newsroom in spreadsheet form – also details difficulties experienced by field staff on census day itself. They reflected problems being highlighted in the media.
In Northland, field staff were “down to three staff out of six”, which forced Stats NZ to phone what’s called non-private dwellings, like hotels and rest homes, to say it wouldn’t get to them that night. Instead, they asked for a list of individuals, including whether they were from overseas. A subsequent note said 822 non-private dwellings had ordered census materials but by March 12, 263 of those hadn’t been sent anything.
Meanwhile, in Auckland, a member of the public rang to say they’d found a pile of census forms, which were retrieved personally by the regional manager, Peter Puchner, and were hand-delivered to addresses the next day.
Problems continued in the following days, including police being called because of a white powder – later confirmed as talcum powder – in a mailed response. Also, some people, sent incorrect access codes, were inadvertently able to see the private data of others.
For Stats NZ, though, a bigger issue was looming, which led it to announce yesterday it was delaying the first release of census data by five months. On March 29, so-called “incident 21” was flagged with census boss Denise McGregor on a “high” priority. It was labelled “response rates too low”.
This year’s data collection was meant to catapult the census into the digital age. The $121 million project was digital-first, meaning 80 percent of people would get online codes instead of paper forms. That meant 4000 fewer field staff would be needed, compared to last time. The census had teething problems with crucial IT systems and paper forms, but management were confident they could pull it off.
McGregor told Newsroom in December with forms coming in and being processed by computer systems, it would show Stats NZ where to focus its efforts. “This time we’ll be actually looking at some process data and saying, well, how are we going for different groups, how are we going in this part of the country, where do we have to redirect the effort. It’s quite dynamic, feeding information that we have to just give intelligence to our effort.”
In particular, McGregor hoped it would allow Stats NZ to lift the count for Maori, which, of all the major population groups, had the highest undercount in the 2013 census, estimated at 6.1 percent. “We’ve put a lot of effort into getting that up,” she said at the time.
Yesterday’s media statement said Stats NZ was delaying its first release of data from October to March of next year because of lower-than-expected response. The department’s “interim calculations” showed at least 90 percent of people had “full or partial” information in the census, down from 94.5 percent five years ago.
That means about one in 10 people didn’t take part, which has been described by University of Auckland professor of statistics Thomas Lumley as a “very serious” drop, The Spinoff reported.
The statement said data for “small populations, subgroups and small geographies” will be “improved” and the quality of census data would be supplemented with administrative data.
There is a long-term, international trend of declining census response rates, the Stats NZ statement said. But that’s not the line McGregor took last December. She told Newsroom this country’s census was modelled on Canada’s, which ran last year and was their “best census ever”. “It ran very smoothly and they got very high coverage and very high internet – very high self-response.”
Three days after census day, contingency money was being chipped in.
Stats NZ’s census action log reveals $35,000 would be spent on “additional comms messaging” in Whangerei and Kaikohe. “$165K left,” said the entry. On March 19, concern was raised that the individual response rate was tracking below its “bottom tolerance line”. That was when Stats NZ admitted publicly that tens of thousands of Kiwis might not have received an online code or paper form by census night.
Additional marketing of $19,000 was approved in South Waikato and Auckland on March 21. A week later, 23 field officers were flown to Auckland. The following week, it was noted “lower than expected paper forms coming in” – beyond what was expected, after ex-cyclone Gita delayed postal deliveries in some areas. Contracts for nine “engagement advisers” were extended for three months.
As Easter approached, census managers were getting nervous. Some territorial authority responses had “flat-lined” and there was mention that Pacific peoples might be a target group for higher responses. All dwellings should have at least one visit by April 2, it was said.
On March 29, Christchurch-based project manager Tracy Moore reported incident 21, “response rates too low”, to boffins running the daily operational meetings. That sparked a chain of discussions over several weeks to try and boost responses.
Field staff didn’t work over Easter, it was decided.
“For some it was the easiest census ever; for others it has been a frustrating experience. For that I am sorry.” – Liz MacPherson
On April 4, the Wednesday after Easter, McGregor extended field work operations until April 22. Spending money on targeted social media ads was discussed, as was dragging in contact centre staff to help. Ideas were mulled about how to use Stats NZ staff to “engage with public”.
It was decided to mount a campaign targeting secure apartments in Auckland and Wellington. Later that week, a 13-strong “flying squad” was sent to Auckland and Hamilton, with 15 more people to arrive on the Saturday. An extra $12,000 was approved for marketing to Aucklanders.
The following week, a boost to small towns with low response rates was considered. Field work in Northland and South Auckland was extended by a week. Discussions about doing “anything extra” on the North Island’s east coast were had on April 16, but it’s unclear whether anything was done.
On June 1, Stats NZ said it had switched off its online census system the previous week. Government statistician Liz MacPherson put a brave face on the results, saying it was pleased with online participation – at 82 percent, well above its 70 percent target.
But some aspects proved more challenging than expected, she said, and it didn’t get everything right. “I have had mixed feedback from people,” MacPherson said. “For some it was the easiest census ever; for others it has been a frustrating experience. For that I am sorry,”
MacPherson’s apology might resonate more deeply with the public if Stats NZ managers are held accountable for the mistakes that have been made – something that might be unclear until a promised full, independent review is complete.