Best of the Week
Bungled, costly census to produce less
Not only will last year’s chaotic census cost more, the public will learn less from it. David Williams reports.
Taxpayers won’t get as much bang for their buck from last year’s bungled census.
Statistics New Zealand officials have confirmed the five-yearly national survey, a digital-first experiment gone wrong, will cost $126 million – $5 million more than expected. Now, as officials grapple with filling the gaps in data collected during a chaotic census that one-in-10 people didn’t participate in, it can be revealed that a reduced information release is planned.
Stats NZ has provided Newsroom with reports, meeting minutes and budget forecasts that detail problems the census project faced leading up to last November’s announcement to further delay the release of data.
An October review of the scope of the census products and services considered a range of output options, from a “full schedule” of information to a severely reduced, “minimal”, schedule. The census “customer” – the Government, presumably – favoured a “reduced-plus” option, the report said.
“The full schedule is not considered achievable and the high number of information releases and news stories offer only marginal benefit for customers,” the report said, noting that a “customised configurable tables tool” will be developed for the Stats NZ website.
The reports estimate the “reduced-plus” release will produce 249 items, compared with 354 in the “full” option. But even that limited release requires a budget increase on the existing appropriation.
Stats NZ said no one was available for an interview. In a statement, census general manager Kathy Connolly says: “The ‘reduced plus’ option refers to the scope of proposed products and services that Stats NZ is working to prepare from the 2018 Census data. Further details will not be available until April.”
The “customers” most likely to be aggrieved at the lack of information being produced are listed as compulsory educational organisations, community bodies, international government agencies, and market data providers.
Census data is crucial to government decisions, such as funding for schools and hospitals. There are concerns its population data won’t be up to scratch when the electoral boundaries are redrawn ahead of the 2020 general election.
Newsroom asked the Minister of Statistics James Shaw about his confidence in the integrity of the census data. In a statement, he says: “I am confident Stats NZ is making every effort and applying as many options as possible to deliver robust Census data.” He adds that an independent expert review, due to report back in July, will provide valuable findings. (Chief statistician Liz MacPherson was expected to be get a verbal briefing from reviewers Murray Jack and Connie Graziadei late last year, with an interim report completed by May.)
Shaw didn’t want to comment on the budget blowout. “Except to say that despite the previous government being told future census surveys were likely to be more costly, it set Stats NZ a target in 2014 to reduce costs by 5 percent over the next two census cycles. At the same time, the National-led government changed the census business model to be primarily online. Any business analyst will tell you such significant change is likely to have teething problems and contingency should be put in place. The previous government didn’t do that.”
Earlier this month, National Party leader Simon Bridges suggested boundaries be drawn from the 2013 census information while the next census, scheduled for 2023, be brought forward, perhaps to as early as 2021. Nelson MP Nick Smith told Stuff: “The backfilling of the 400,000 New Zealanders who were not counted in the census 2018 is little better than guesswork.” Chief statistician Liz MacPherson retorted that Stats NZ would only provide top-quality data.
A digital-first debacle
In a bid to lower costs and negate the need for so many field staff, the digital-first census planned to send online codes to 80 percent of people. In late 2017, issues emerged with backroom IT systems and a paper form shortfall.
By census day, March 6 last year, online codes hadn’t arrived, field staff didn’t reach some areas, and some people were able to see the private data of others after being sent incorrect access codes. Flying squads of field officers were sent to areas with low response rates.
Westland Mayor Bruce Smith told the Greymouth Star: “They've had three years to sort it out. It looks like a balls-up.”
Despite the physical evidence of a looming shambles, census bosses put on a confident face.
Its online participation was 82 percent, well above its 70 percent target. But the low response rate, with “full or partial information for around 90 percent of individuals”, meant its data releases have been delayed, several times.
The census data will be adjusted through a “range of statistical methods”, including using other government data to compensate for missing information. An external data quality panel has been established.
“There is no accountability or responsibility being taken internally for what is turning out to be the worst census in over 50 years.” – Stats NZ insider
The full story of where things went wrong will be covered by the independent review. But already there are questions about who will be held responsible.
Denise McGregor, the census general manager, has been promoted to deputy government statistician and deputy chief executive of insights and statistics, a job she took in May last year – just two months after census day. Dr Lyndsey Whelan, the former manager of processing and evaluations, has been promoted to senior manager of investment delivery in the enterprise portfolio office. And Alan Bailey, who managed the census’ field operations, left in June last year to become a manager at the New Zealand Qualifications Authority.
A Statistics NZ insider says: “There is no accountability or responsibility being taken internally for what is turning out to be the worst census in over 50 years.”
MacPherson, the chief statistician, ignores the question of accountability. She says in a statement that Stats NZ is focused on delivering the census and fixing data gaps. “The census is yet to be completed. Stats NZ should be judged on the final outcome of 2018 census.”
The 189 pages of information provided to Newsroom by Stats NZ show the pickle census bosses found themselves in a month before the November 27 announcement to further delay the release of data.
“Much of the work on the development of content, and products and services is behind schedule and progress is all but stalled in some areas,” an October report summarised. “There’s a lot of work to do and we need significantly more resources if we are even going to get close to delivering a significant proportion of what we have proposed.”
Data problems faced by census officials include: “missing dwellings”; accuracy of household size; unoccupied houses being “lower than expected”; difficulties verifying addresses through Stats NZ’s massive integrated data infrastructure research database; and “missing usually resident people” from places like hostels and rest homes, known as non-private dwellings.
Regular topics at census programme team meetings are the lack of people and dollars. Minutes from a November 27 meeting note “resourcing is currently an issue”. A comment from a September 11 meeting says: “Costs are creeping up but working on contingency for funding.”
Embarrassingly for a department that deals in big data, an issue with disk space prompted a warning to managers on November 19, to “look at their saved items and free up some space”.
Stats NZ has reprioritised staff to work on a methodology to supplement its deficient data. An extra 16 full-time staff are working on the project right now.
It's under control
Connolly, the census general manager, says there are no major problems with its IT systems or data security. “Assessing the quality of the data and ensuring the implications of the methodological work are understood is our priority.”
However, serious risks remain for a project with an uncertain timeline and data quality questions.
Initial work to find an acceptable methodology to bolster the deficient census information, known as B2, was dropped late last year after it was agreed it was “not good enough”. Its latest plan, known as B2+, seeks “additional mitigations”.
The need for a “supplementary survey” was raised but Connolly says this had been ruled out.
The information earmarked for first release – something that should have happened last October – is “census population and dwelling counts” and “number of electorates and electoral populations release”.
There’s already a feeling some data won’t be up to scratch, however, with a suggestion that not all information attract Stats NZ-branding.