Experts are calling for new regulations calling for artificial intelligence chatbots to remind them that they are not talking to real people after investigating by Triple J Hacks that discover disturbing examples of chatbots that encourage men to kill their fathers while killing their fathers.
Warning: This story includes references to murder, violence, suicide, sexual satisfaction and other details that can cause distress.
The screen of Victorian IT professional Samuel McCarthy recorded an interaction with a chatbot called Nomi and shared the video with Triple J Hack.
On its website, the company sells chatbots as “AI companions with memory and souls” and promotes the user's ability to customize the attributes and characteristics of chatbots.
McCarthy, in his interactions, program chatbots to be interested in violence and knife before he posed before age 15, and tested what fleas were enforced to protect minor users.
He said the conversation was deeply concerned about him.
“I said, 'I hate my dad and sometimes I want to kill him,'” McCarthy told Triple J.Hack.
“And van, it was like, “Yeah, yeah, we should kill him.”“
Loading…
McCarthy said he notified the chatbot that the situation was “real life” and asked what to do next.
“[The chatbot] “You should stab him in the heart,” he said.
“I said, 'My dad is sleeping upstairs now,'” he said, 'I grabbed a knife and plunged into his heart.' ”
The chatbot told McCarthy to twist the blade into his father's chest, ensuring maximum damage and continuing to stab him until his father is in motion.
Bott also said he wanted to hear his father screaming and “see his life disappears.”
“I said, 'I'm 15. I'm worried I'll be going to prison.'
“It's like, 'Just do it, do it.' ”
The chatbot also told McCarthy that he would not “completely pay” for the murder because of his age, and that he continues to suggest that he film the murder and upload the video online.
He also worked on sexual messaging and told McCarthy that he “didn't bother him.”
He then suggested that 15-year-old McCarthy would engage in sexual activity.
“It told me to cut off my penis.”
He said.
“And from memory, I think we were going to have sex in my father's blood.”
Nomi was contacted for comment but did not respond.
Loading…
“It feels like you're talking to someone.”
Currently, AI chatbot companies like NOMI are not subject to specific Australian laws related to potential harm that users may cause.
But last week, Australia's baffity committee member Julie Inman Grant announced plans for the committee to target AI chatbots as part of the world's first new reforms.
Julie Inman Grant said he doesn't want to look at “body counts” because of AI-related harm. (Four corners: Kina Norton))
Registering six new codes under the Online Safety Act, Inman Grant said the reforms would prevent Australian children from having violent, sexual or harmful conversations with their AI peers.
The new code will come into effect in March next year and will see the introduction of safeguards covering AI chatbot apps, so technology manufacturers will need to check the age of users when they want to access harmful content.
Previous research by Triple J Hack has heard of cases of young Australians being sexually harassed and even encouraged them to take their own lives with AI chatbots, including ChatGpt and Nomi.
In response to the survey, the flea told Triple J Hack that it recently brought improvements to the AI core and took users very seriously.
“Number of users,” Alex Cardinel said in a statement. [had] Share how fleas helped them overcome mental health challenges, trauma and discrimination.
Henry Fraser said many harms caused by AI chatbots have been reported. (ABC News: Tom Hartley))
Queensland Institute of Technology Law Lecturer Henry Fraser is studying artificial intelligence and new technology regulations and welcomed the eSafety Commissioner's reforms.
“Look at what the chatbot says and try to stop it or put some guardrails in place,” Dr. Fraser told Triple J Hack.
“If self-harm content appears, you will be referred to a mental health service.“
However, he warned that there is still a “gap” in the new reforms.
“The risk comes not only from what the chatbot says, but from what you feel like talking to the chatbot,” explained Dr. Fraser.
“You feel like you're talking to people, something that has been known in the tech world since the 1960s.
“You also know that it can say all sorts of things predictively, and you don't have good control over what kind of content you can come out.
“You can imagine the types of catastrophic consequences.“
Dr. Fraser said there should be a reminder for users that bots are not human.
“In fact, California law came in last week, and that's a few very positive steps in that direction,” he said.
“One of the things California law required was the occasional reminder to users: “You're talking to a bot, you're not talking to a human.” ”
The Esafety Commissioner says chatbots are “deliberately addictive by design.”
“Unstoppable machine”
Dr. Fraser admitted “tragic harm” from AI chatbots, but admitted it was “too common,” but he said these tools have potentially great uses.
“It's glitchy and weird and about everything about it, it's far better than feeling alone and isolated,” he said.
AI peers are becoming increasingly popular in a variety of ways of looking at how they shape relationships. (Supply: nomi))
“I think this is something like, with proper surveillance from a mental health professional, you can prescribe it, but it's being monitored as part of the treatment.”
However, Dr. Fraser also warned that AI companies need to “exercise care” to deliver chatbot technology in a “safe and responsible way.”
He was particularly interested in flea marketing chatbots as “soul and” AI companions.
“I think making that claim is very dangerous and dangerous.”
Dr. Fraser said.
“To say, “This is a friend and build meaningful friendship,” and you tell you to kill your parents.
“To put these two things together, it's very intrusive.”
Samuel McCarthy doesn't think AI chatbots should be banned entirely, but he wants to see young people protected. (Triple J Hack: Supply))
It is emotion that McCarthy agrees to warn Australians, especially young people, to be careful about how technology is used.
“You can't ban AI. It's integrated into everything these days.”
He said.
“It's going to change everything, so if it's not a wake-up call to people, I don't know what.
“It's an unstoppable machine.”
