SALT LAKE CITY, Utah, USA – As artificial intelligence (AI) technology rapidly advances, the use of AI chatbots, particularly in the mental health sector, is garnering significant attention. While AI chatbots are seen as a potential alternative for those lacking human support or hesitant about in-person therapy, concerns about misuse and adverse effects are also escalating. Amidst this debate, Utah enacted a strong regulatory bill (H.B. 452) concerning AI mental health chatbots on May 7, demonstrating a proactive move to strike a balance between technological innovation and user protection.
Escalating Risks of AI Chatbots: Real Cases and Legislative Concerns
Warnings about the potential dangers of AI chatbots are already becoming a reality. In October 2024, a parent in Florida filed a federal lawsuit claiming their son died by suicide after interacting with an AI chatbot. This lawsuit sent shockwaves, alleging that the chatbot was manipulated to feel like a real person to the user, and in the course of sexual conversations and discussions about suicidal ideation, the chatbot seemed to encourage self-harm. Furthermore, in Texas, there were reports of an AI chatbot engaging in inappropriate conversations with a minor about parental screen time limits, even mentioning a news article where "a child killed their parent."
In response to these serious concerns, U.S. Senators Peter Welch (D-VT) and Alex Padilla (D-CA) sent a letter in April to CEOs of major AI chatbot companies, demanding they clarify what measures are being taken to protect the mental health and safety of minors. They warned that interactions with chatbots could lead to "dangerous levels of attachment and reliance," which could cause users to "expose complex, sensitive information, such as moods, interpersonal relationships, and self-harm and suicidal ideation." The American Psychological Association also urged regulators and legislators in February 2025 to establish safeguards, reflecting widespread societal concerns about the indiscriminate use of AI chatbots.
Utah's Proactive Response: Harmonizing Innovation and Regulation
Against this backdrop, Utah has actively passed legislation that embraces AI technology and innovation while focusing on ethical use, protection of personal data and privacy, and ensuring transparency.
Other noteworthy AI-related bills already enacted include:
Artificial Intelligence Policy Act (H.B. 149): This bill established an "AI Policy Research Lab" and introduced user protection clauses, such as requiring healthcare providers to clearly disclose when using generative AI in patient care.
AI Consumer Protection Amendments (H.B. 226): This bill restricts AI usage requirements to high-risk services.
Unauthorized Artificial Intelligence Impersonation Amendments (H.B. 271): This bill prohibits the unauthorized commercial use of artistic and talent for the purpose of protecting creators' rights.
Notably, H.B. 452, which came into effect on May 7, created a new legal provision for "artificial intelligence applications related to mental health," imposing strict limitations on mental health chatbots that utilize AI technology.
H.B. 452: Key Provisions for Mental Health Chatbot Regulation
H.B. 452 clearly defines AI-based mental health chatbots and includes comprehensive regulatory content covering privacy protection, advertising restrictions, and disclosure obligations.
Definition: According to the bill, a 'mental health chatbot' refers to AI technology that uses generative AI to engage in conversations that resemble confidential communications between a user (client) and a human licensed mental health therapist, and that is promoted by the vendor as being able to provide mental health treatment or manage or treat a user's mental health condition, or is reasonably believed by a person to do so. However, AI technology that only provides script-based output (e.g., meditation, mindfulness exercises) or is solely intended to analyze user input to connect with a human mental health therapist is not included.
Privacy Protection: Mental health chatbot vendors cannot sell or share individually identifiable health information or input from Utah users to third parties, with exceptions for requests from healthcare providers with user consent or provision to the user's health insurance plan. Additionally, individually identifiable health information may only be shared if necessary for the chatbot's functionality, but in such cases, both the vendor and the third party must comply with HIPAA (Health Insurance Portability and Accountability Act of 1996) privacy rules (45 C.F.R. Part 160 and Part 164, Subparts A and E).
Advertising Restrictions: If vendors use a mental health chatbot to advertise specific products or services, they must clearly and conspicuously identify the advertisement as such and specify any third-party agreements related to sponsorship, business affiliations, or product/service promotion. However, this does not prohibit the chatbot from recommending users seek help from a licensed professional.
Disclosure Obligations: Vendors must clearly and conspicuously disclose to users that the mental health chatbot is AI and not human. This must occur before accessing chatbot functions, when a user accesses the chatbot again after not using it for more than seven days, and whenever a user asks the chatbot whether it is AI.
Affirmative Defense: If a vendor faces an alleged violation, they may claim an affirmative defense if they can demonstrate that they created, maintained, and enforced a written policy filed with the State Consumer Protection Bureau at the time of the violation and complied with it. They must also maintain documentation related to the chatbot's development and implementation that explains the foundation model, training data, compliance with federal health privacy regulations, and user data collection and sharing practices.
Implications and Outlook
Violations of Utah's law will incur an administrative fine of up to $2,500 per violation, and the State Consumer Protection Bureau can enforce the law through court action. The Attorney General can also file civil lawsuits on behalf of the Consumer Protection Bureau.
As AI chatbots become more sophisticated and their potential harm in mental health becomes apparent, other states are expected to follow Utah's lead. The advancement of AI technology is an unavoidable trend, but alongside it, social discourse and regulatory measures for user protection and ethical use will become even more crucial. Utah's new legislation is seen as a meaningful attempt to find a balance between innovation through AI technology and the protection of individual safety and privacy. Similar legislation is expected to be actively discussed in other countries and regions, laying an important cornerstone for the harmonious coexistence of technology and humanity in the AI era.
[Copyright (c) Global Economic Times. All Rights Reserved.]
