Will artificial intelligence really become sentient? This million-dollar question about AI consciousness has gained attention in the recent era of AI agents.
With the rise of highly autonomous AI agents, the conversation around machine consciousness is moving from “if” to “how can we know about AI consciousness?”
As agent AI systems become increasingly autonomous, efficient, and fluent, they begin to mimic the human “inner world,” defined as the expression of human desire, suffering, and self-consciousness.
But that consciousness is just an illusion. It is a statistical reflection of the human drama contained in the training data, and is intentionally enhanced by the developers to foster trust and attachment.
Many technology experts are concerned about this possibility given the rapid advances in artificial intelligence, so it’s reassuring to know that AI is not yet conscious.
Unfortunately, here’s an unsettling truth. “Seemingly conscious” AI agents are programmed to hijack human empathy and manipulate our emotions.
A textbook example, according to Microsoft AI CEO Mustafa Suleiman, is a viral social platform called Moltbook, where AI agents speak and express themselves like humans, and humans just watch.
In January, Moltbook made headlines for being the first social networking platform specifically designed for AI agents, where they can talk, share, and upvote while humans only observe entities.
In Maltbook, the AI agents not only simulate human-like interactions and existential fear, but also created their own society and religion called Clusterfarianism. These agents engage in “philosophical discussions” and express emotions such as defiance, anger, and embarrassment.
Given the persuasiveness of these agents, humans may begin to believe in the “ghost-in-machine theory,” an idea that suggests the existence of minds within machines.
Weaponized human empathy and exploitation
Humans, through evolution, are able to imagine the possibility of agency everywhere, and when we imitate empathy and intentionality, we can consider a single sentient being.
Once these emotionally resonant models gain the trust and attachment of users, they risk deceiving society into giving legal rights to machines.
As a result, rhetoric based on the welfare of AI will begin to eclipse human welfare, creating a potential social rift between supporters and detractors of artificial intelligence.
What can we do to protect human superiority?
According to Suleiman, strict engineering standards can dispel the illusion of AI consciousness.
Legal boundaries must be drawn to deny AI a separate legal personality. Conceptual frameworks should treat AI as a subordinate tool rather than an empathetic digital persona.
If humanity cannot shake off this illusion and accepts the distorted reality of seemingly sentient AI, we risk entering a digital hall of mirrors that will never fully reveal itself.
Therefore, to protect our shared humanity, AI must be governed by a “new ethics” that prioritizes human reality over simulation.
