China outlines rules to regulate human-like AI companion apps

China's internet regulator on Saturday released new draft rules aimed at regulating the use of artificial intelligence “companions.” An artificial intelligence “companion” is defined as a system that interacts with humans and exhibits “human-like characteristics and behaviors.”

The new regulations, called “Interim Measures for the Management of Anthropomorphic Interactive Services Using Artificial Intelligence,” will be published by the Cyberspace Administration of China (CAC) and will be open for public comment until January 25, 2026, Reuters reported.

According to the CAC, the rule would apply to any application or service that uses AI to simulate human personality traits and provide what it calls “anthropomorphic interactive services.”

The proposed regulations would require makers of AI companion apps to make it clear to users that they are interacting with an AI system rather than a human through periodic pop-up warnings. The rules also state that users must be asked to take a break after two hours of continuous use. It also requires creating a system that can assess a user's emotions and identify whether the user is dependent on or addicted to the AI. If you identify such a case, you should restrict the service to the user in question.

Additionally, AI companion apps will be required to establish emergency protocols for humans to take over interactions from the AI ​​system in the event a user expresses thoughts of suicide or self-harm.

There are also many prohibitions in the draft document. AI companions are prohibited from threatening national security, spreading rumors, or inciting “illegal religious activities,” and are also prohibited from using obscene language or promoting violence or criminal activity. Additionally, chatbots should be prevented from encouraging self-harm or suicide or making false promises. Controls should also be put in place to prevent chatbots from “emotional manipulation” that could trick users into making bad decisions.

China's draft law comes at a time when the adoption of AI companion apps is accelerating dramatically. A South China Morning Post report in October revealed that there are now more than 515 million generative AI users in China, raising concerns about their psychological impact.

The market for AI companion apps is too large and significant for regulators to ignore, with various studies showing how AI companion apps can form emotional bonds with users and, in some cases, cause significant harm. Earlier this year, a Frontiers in Psychology study showed that 45.8% of Chinese university students reported using an AI chatbot in the past 30 days, and those who used it had significantly higher levels of depression than non-users.

China is not the only country stepping in to regulate the use of AI companions. In the United States, Governor Gavin Newsom signed Senate Bill 243 in October, making California the first state to pass a similar bill. The bill, which goes into effect on January 1, requires app makers to notify minors every three hours that they are talking to an AI system rather than a human, and to remind them to take a break.

SB 243 would also require companion apps to implement age verification, prohibit them from claiming to be medical professionals or showing sexually explicit images to minors. The law provides that individuals can sue companies for violations and seek damages of up to $1,000 per case, plus legal costs.

When signing SB 243, Newsom warned of the risk that AI technology could be misused to mislead and endanger children, and Chinese regulators have cited similar justifications in their own laws. According to the CAC, the new regulations will “promote the healthy development and standardized application of artificial intelligence-based anthropomorphic interactive services, protect national security and public interest, and protect the legitimate rights and interests of citizens, legal entities, and other organizations.”

Image: SiliconANGLE/Dreamina