Big data claims starting to fray

The Government's flagship policy of using data mining to surgically direct social investment to beneficiaries through non-government organisations received two hammer blows in 24 hours - one self-inflicted and one from the Privacy Commissioner

On Wednesday, Social Development Minister Anne Tolley revealed that a privacy hole had been discovered in a computer portal set up to collect data about people receiving assistance from social service providers.

The privacy breach was in a portal used by non-government organisations receiving Ministry of Social Development (MSD) funding who were requested to upload highly confidential client information. One organisation discovered it was able to view another provider’s folder, containing the confidential information, which led to the portal being shut down on Tuesday night.

The second hammer blow came from Privacy Commissioner John Edwards the morning after Tolley's announcement. He released a report into the data sharing by service providers which found the MSD policy was "excessive and inconsistent".

The report looks at the privacy impact of the funding contracts, which make the provision of this highly personal, identifiable, client data a requirement for receiving government funding, with no ability to ‘opt out’.

Edwards said the report found there had been insufficient consideration given to the possible unintended consequences of the policy change, and insufficient consideration of alternative means of achieving the Government’s legitimate aims without risking those consequences.

"There is a real risk that the new arrangement will deter some people who are most in need from seeking support or assistance," Edwards said.

Over-selling the big data supermarket?  

It has made for a painful week for both Tolley and the Prime Minister, as for years "big data" has been the favourite theme of Bill English in speeches to business and other groups.

Presented as the Government's central and big new idea to address social problems, data mining is the key to the much-vaunted 'social investment approach', which English claims will see Government targeting its spending on programmes to break inter-generational dependency much more effectively.

In a speech at the third Data Hui last year, the then Finance Minister described the Government's plan to provide a "data supermarket" for service providers.

He said that access to data on people needing assistance was too limited: "the door is small, it's a bit hard to find, and when you do get it you'll only get the food you're given. But what we want is a supermarket," he said.

"The kind of supermarket with a door that is easy to find, and that has rules for entry that are strict but clear. And once you’re inside this supermarket, it’s you who gets to choose what you want. And you can be confident that on your way out you aren’t going to be stopped by security second-guessing what you might end up cooking."

"For social service providers, this will make data access professional, routine, reliable and responsible – rather than a negotiation," he said then.

Now it is looking like that supermarket door was left too far open.

Even before his report on the data sharing policy, the Privacy Commissioner was sounding the alarm about the risks of talking up the promise of  'big data'.

He told Newsroom last week that he wanted to offer the Government "a word of caution about overconfidence" in its data mining and social investment claims.

"To say 'Well, you say you’re going to get that result, are you sure?' There’s a lot of snake-oil salespeople out there promising that data can have your baby or that the blockchain is going to save the world – it’s not true, there’s a lot of overselling of this stuff," Edwards said.

“I worry about a level of overconfidence about what the technology can deliver.”

Help us create a sustainable future for independent local journalism

As New Zealand moves from crisis to recovery mode the need to support local industry has been brought into sharp relief.

As our journalists work to ask the hard questions about our recovery, we also look to you, our readers for support. Reader donations are critical to what we do. If you can help us, please click the button to ensure we can continue to provide quality independent journalism you can trust.

With thanks to our partners