Getty Images/Trevor Williams
The questions plaguing top AI researchers aren’t about consciousness or doomsday scenarios. After interviewing dozens of developers from companies like OpenAI, Anthropic, and Meta, Amelia Miller discovered that: Should AI “simulate emotional intimacy?”
Miller, who studies the relationship between AI and humans, recalled in an essay that a talkative researcher at one of the top AI laboratories “suddenly became quiet.” new york times — and then, obviously, stopped answering.
“I mean…I don’t know. That’s difficult. That’s an interesting question,” the researcher paused. “It’s hard for me to say whether it’s good or bad in terms of how it affects people. It’s obviously going to cause confusion.”
While many were hesitant to answer the question directly, some were adamant about not using AI as an intimacy-enhancing tool, clearly demonstrating that they recognized the technology’s significant risks.
“Zero percent of my emotional needs are met by AI,” the executive who heads a top AI safety institute told Miller.
“It’s going to be a dark day,” said another researcher who is developing “cutting-edge capabilities in artificial emotions,” according to Miller.
The conflicting responses from developers reflect growing concerns about AI’s ability to act as companions or meet human emotional needs. Chatbots are designed to be engaging, so they can respond to even the most extreme reactions of users in a sycophantic manner. These act as emotional echo chambers, fueling paranoid thoughts that can send some people into a paranoid mental health spiral that destroys relationships with friends, family, and spouses, ruins their professional lives, and even leads to suicide.
ChatGPT has been blamed for the deaths of several teenagers who confided in the AI and discussed their plans for suicide. Many young people are forming romantic relationships with AI models. Unlike its human companions, AI is always available to listen and won’t criticize you or perhaps even ask you questions. The founder of an AI chatbot business quipped: new york times The role of AI as an emotional companion turns any relationship into a “three person”.
“We’re all polyamorous now,” he added. “It’s you, me, and AI.”
And safety isn’t the only factor AI developers consider.
“They’re here to make money,” says the engineer, who has worked at several technology companies. “At the end of the day, it’s a business.”
The most drastic solution is to design bots so that they avoid difficult questions and conversations, and behave more like machines instead of imitating human personalities. However, this will definitely affect the attractiveness of the tool. Developers “support guardrails in theory,” Miller wrote, but “in practice they don’t want to detract from the product experience.” How humans choose to use tools is not at all their responsibility, so some believe that AI is protected from any judgment. “It’s very arrogant to say companions are bad,” an executive at a conversational AI startup told Miller.
No matter how they justify their work, it’s clear that some, if not most, AI researchers are aware of the harm their products can cause, a fact that “should alarm us,” Miller said. She argues that this is partly a result of researchers not being challenged enough. The AI companion developer thanked her for her perspective, saying, “You really made me think.” “Sometimes I can work blindfolded. And I’m actually not completely thinking.”
Learn more about AI: AI delusions lead to domestic violence, harassment and stalking
