Call for safety measures to prevent unwanted “stalking” of deceased loved ones by AI chatbots

Without safety standards in its design, artificial intelligence that allows users to communicate with deceased loved ones via text or voice could cause psychological harm or harm those left behind by using digital technology, according to researchers at the University of Cambridge. There is a danger that they may even be “followed” by others.

A “deadbot” or “griefbot” is an AI chatbot that uses the digital footprints left behind by a deceased person to simulate their language patterns and personality traits. Some companies already offer these services, offering an entirely new type of “presence after death.”

AI ethicists at Cambridge's Leverhulme Center for the Future of Intelligence have outlined three design scenarios for platforms that could emerge as part of a developing 'digital afterlife industry'. It shows the potential consequences of careless design in the field of AI, which they describe as “high risk.” . ”

Research published in journals Philosophy and technologycompanies could use deadbots to secretly promote products to users as if they were a deceased loved one, or to torment children by claiming their deceased parent is still “with you.” It emphasizes gender.

When a living person signs up to be virtually recreated after death, their chatbots are used by companies to spam surviving family and friends with unsolicited notifications, reminders, and updates about the services they provide. This is digitally similar to “dead stalking”.

Researchers argue that even people who initially derive comfort from deadbots can become depleted by day-to-day interactions that carry an “overwhelming emotional weight,” but if a deceased loved one is If the company has a long-term contract with a company, there is a possibility that the AI ​​simulation will be interrupted and the company will be powerless. Digital afterlife services.

“Rapid advances in generative AI now mean that almost anyone with internet access and basic know-how can bring a deceased loved one back to life,” said study co-author Dr. said Dr. Katarzyna Nowaczyk Bazyńska, a researcher at the Leverhulme Center for Future Intelligence. (LCFI).

“This area of ​​AI is an ethical minefield. It is important to prioritize the dignity of the deceased and ensure that this is not compromised by financial incentives such as digital afterlife services. ” AI simulation as a farewell gift to a loved one who is not ready to process grief in this way is an equal right of both the data provider and the person using the AI ​​postmortem service. Should be protected. ”

There are already platforms that offer AI reconstruction of the dead for a small fee, such as Project December, which started using the GPT model before developing its own system, and apps like HereAfter. Similar services are starting to appear in China.

One possible scenario in the new paper is MaNana, a conversational AI service that allows people to create deadbots modeled after their deceased grandmothers without the consent of the “data provider” (deceased grandparents). Thing.

In a hypothetical scenario, an adult grandchild who is initially impressed and reassured by the technology begins receiving ads once the “premium trial” ends. For example, a chatbot that suggests food delivery orders in the voice and style of a deceased person.

This relative feels they are disrespecting their grandmother's memory and wants Deadbot to be turned off in a meaningful way that the service provider does not consider.

Co-author Dr Tomasz Horanek, also from LCFI at the University of Cambridge, said: “People may develop strong emotional bonds through these simulations, making them particularly vulnerable to manipulation.”

“We need to consider methods and even rituals to retire the deadbot with dignity. This could mean, for example, a form of digital funeral or other types of rituals depending on the social context. We recommend designing protocols to prevent deadbots from being exploited in disrespectful ways, such as through advertising and having an active presence on social media. ”

While Hollanek and Nowaczyk-Basińska argue that recreational service designers should proactively seek consent from data providers before passing through, the ban on deadbots based on non-consensual providers is They claim that it is not possible.

They say the design process will include a series of prompts for people wishing to “resurrect” a loved one, such as “Have you talked to X about how they would like to be remembered?” I suggest that you should. Therefore, the dignity of the deceased is paramount in the development of Deadbots.

Another scenario featured in the paper, a fictional company called “Paren't,” highlights an example of a terminally ill woman leaving a deadbot behind to help her 8-year-old son through his grieving process. .

The Deadbot initially serves as a therapeutic aid, but the AI ​​begins to generate confusing responses as it adapts to the child's needs, such as depicting an impending face-to-face encounter.

The researchers recommend age restrictions for deadbots and call for “meaningful transparency” so users always know they are interacting with an AI. These may be similar to current warnings about content that may cause seizures, for example.

In the final scenario considered in the study (a fictitious company called “Stay”), an elderly person secretly takes care of his own dead body in hopes of comforting his adult children and hoping that his grandchildren will be well. It shows that you are committed to the bot and have paid a 20-year subscription fee. know them.

After death, services begin. One of her adult children, uninvolved, receives a flood of e-mails in the voice of her deceased parent. The other one does, but ends up emotionally exhausted and racked with guilt over the fate of the Deadbots. However, suspending a deadbot would violate the terms of its parent's contract with the service company.

“It is critical that digital afterlife services consider the rights and consent of not only those reenacting, but also those who need to interact with the simulation,” Horanek said. “These services run the risk of causing great suffering to people if they are exposed to unwanted digital possession by incredibly accurate AI recreations of those they have lost. The impact could be devastating, especially during an already difficult time.”

Researchers are calling on design teams to prioritize opt-out protocols that allow potential users to end their relationship with deadbots in a way that provides emotional closure.

Nowaczyk-Basińska added: “We need to start thinking now about how to mitigate the social and psychological risks of digital immortality, because this technology already exists.” .