Proposed bill could allow government to use AI to make decisions about people’s interests

The proposed new law would allow the Ministry of Social Development to use AI to make decisions about people’s interests.

Changes to the Social Security Act are being urgently discussed in Parliament, but this means bypassing the select committee process, which includes public consultation and scrutiny.

The law already allows for “targeted” use of “automated decision-making,” and if passed, the bill would expand its scope even further.

This would allow MSD to “authorize the use of automated electronic systems by specified persons to make decisions, exercise powers, comply with obligations, or take other related actions, subject to appropriate safeguards and subject to specified provisions.”

MSD told RNZ that the proposed changes do not include the use of generative AI such as ChatGPT.

A memo attached to the bill says it aims to “improve the efficiency and effectiveness of the administration of the welfare system.”

National’s Scott Simpson introduced the bill, saying MSD makes millions of decisions each year and staff spend a significant amount of time managing them.

“This is not good enough for MSD customers or taxpayers, and this bill fixes that.”

He said the technology would be used “judiciously”.

“Automated decision-making will be used to make decisions based on simple rules, with human judgment remaining where necessary,” he said.

“This means faster decision-making, more consistency, and a system that people can trust.”

Labour’s Helen White said the regulatory impact statement outlining the aims, costs and benefits of the law had been redacted in the section outlining the problems it was trying to solve, “so it’s very difficult to know what’s going on here”.

People should be able to expect human contact to be part of their welfare system, she says.

“You’re talking about the very group of people who are the most disconnected. It’s so important that we protect that connection.”

Her colleague Ingrid Leary said automation was the government’s way of replacing staff who may lose their jobs after the Budget.

Ricardo Menéndez March of the Green Party said the move was a significant expansion of powers.

“This is basically a completely blank slate extension of allowing robots and machines to impact people’s lives,” he said.

They referred to the Australian government’s automated system known as robo-debt, which mistakenly asks welfare recipients to repay their benefits.

A subsequent investigation found that it made the victim feel like a criminal, leading to suicide.

“People have died in Australia because of automated systems that have ruined people’s lives, made mistakes and left them in debt,” Menendez-March said.

He said it was “extremely concerning” that the bill was passed without consultation or scrutiny.

MPs from coalition parties spoke briefly, as is often the case in emergencies, in an effort to speed up the process.

New Zealand First’s Jamie Arbuckle said the bill was “an important step towards a more efficient and modern welfare system that serves both taxpayers and those in genuine need”.

ACT MP Parmjeet Parmar said her party supported it for the same reasons.

She argued that staff would have more time to help get people off benefits and into work, and appropriate safeguards would also be put in place.

“Of course there will be humans there to help…nobody takes away that human element,” she said.

But Te Pati Māori MP Orini Kaipara questioned the safeguard claims.

“We’ve heard it before… we heard it when the department illegally targeted beneficiaries through debt collection, we heard it when Māori people were unfairly sanctioned, we heard it when our disabled community, the Waikaha, were repeatedly forced to prove they were still disabled,” she said.

“Technology is not neutral if the system itself is unequal.”

The bill also would expand the list of benefits that are subject to periodic reviews to ensure people remain eligible and receive the appropriate amount.

It would also require beneficiaries to provide medical evidence for some benefits and exclude children from caregiver benefits once they turn 18.

Sign up for Ngā Pitopito Korero. Our daily newsletters, hand-picked by our editors, delivered straight to your inbox every weekday.