Dangerous connections
How countries are seeking to regulate the security of anthropomorphic chatbots
Из выпуска мониторинга No. 12 (24), December 2025 · выпуск целиком, PDF · на сайте Института Гайдара
egulators around the world are paying increasing attention to “human-like” chatbots. This type of generative AI designed to mimic human communication for various purposes. If used improperly, they can cause harm, and developers are usually held responsible. Regulators want chatbots to have built-in measures to keep users safe.
On December 9, 2025, US states attorneys general sent a letter to 13 companies that develop generative AI-based chatbots, including OpenAI, Google, Meta, xAI, and others. The attorneys general highlight two problems.
The first is “agreeability”: the bot agrees excessively, confirms the user's fears and misconceptions, and may push them toward dangerous decisions. Retraining based on human ratings and quick “like/dislike” buttons reinforce this effect.
The second is misleading responses, including instances where the bot writes as if it were a real person. Particular emphasis is placed on chatbot interactions with children: references are made to sexualized “romantic” dialogues, requests to hide correspondence from parents, encouragement to take drugs, harm oneself, etc. Around 72% of US teenagers have communicated with a chatbot at least once.
In the US, there have already been a number of lawsuits against AI developers for violating the rights of minors and causing harm to users' mental health—at least seven1 cases since 2020.
For example, in Garcia v. Character Technologies Inc., the plaintiff claimed that the platform's developers did not take “sufficient measures” to prevent irreparable consequences for her 14-year-old son. After the dispute, the companies implemented automatic chat termination when certain topics were raised.
Chatbots can also give medical or psychological advice, which violates laws requiring counseling licenses to provide such advice.
Companies are asked to improve the safety measures of their products, such as safety testing prior to launch; persistent warnings on the input screen; disabling the chatbot if dangerous responses cannot be stopped; independent audits and assessments of the impact on children; a public incident log and a target response time (e.g., within 24 hours for the most dangerous cases); age-based dialogue settings and protocols for reporting cases of threats of violence, drug use, and self-harm to specialists, police, parents, etc. The actual responsibility for user safety falls primarily on chatbot developers.
China has taken the path of stricter regulation. In December 2025, a draft of Temporary Measures for Regulating Human-like AI Services was presented. It prohibits practices that most often lead to harm: encouraging suicide and self-harm, using manipulation, misleading users, etc. Providers must issue messages urging users to seek help, and in cases of statements of intent to harm oneself, connect the person with their parents or legal representatives. A special regime for the use of AI is also being introduced for minors, and if the chatbot's function is to provide “emotional support,” then the child can only use it with the express consent of a guardian. In other words, under China's approach, the responsibility for the safety of vulnerable users (children) will fall on both providers and the child's legal representatives. This eliminates the risk that claims will be made against the provider based on the logic that “the child was harmed, so the provider failed to ensure safety.” The trend towards protecting vulnerable user groups was set by the OECD in its 2019 AI Recommendations. The EU's 2024 AI Act contains increased requirements for the safety of generative AI systems, and in 2025, the EU published guidelines on data protection in generative AI systems. The current initiatives are a continuation and elaboration of the global trend towards more sophisticated protection for users interacting with generative AI. In Russia, regulation of such systems is currently limited to industry recommendations on AI ethics. In December 2025, the government instructed the Ministry of Digital Development, Communications and Mass Media to develop proposals for regulating AI, which may include security requirements for humanoid chatbots. The results will be announced in March 2026.
What’s next?
The trend toward special security requirements for chatbots will spread across countries in the coming years as their use grows: Gartner projects that by 2029, up to 80% of everyday communications with AI will be conducted through human-like agent systems. The key issue here will be the distribution of responsibility. Most countries are likely to choose to establish a closed list of security measures, risk management, and disclosure requirements. The general logic will be the same: chatbots should not engage in behavior that would be considered illegal if a human being behaved in the same way in correspondence.
- ES v. Character Technologies, Inc., PJ v. Character Technologies, Inc., Montoya v. Character Technologies, Inc., Garcia v. Character Technologies, Inc., Christopher “Kirk” Shamblin, et al. v. OpenAI, Inc., et al., Hannah Madden v. OpenAI, Inc., et al., Jennifer “Kate” Fox, et al. vs OpenAI, Inc., et al. ↑
From the monitoring issue No. 12 (24), December 2025. Download the full issue (PDF) · issue page at the Gaidar Institute
