In early August, Illinois enacted the Psychological Resource Health and Oversight Act (HB 1806, or the “Act”), becoming the first state to pass legislation regulating the use of AI.[1] In the provision of therapy and psychotherapy services. Effective immediately, the law imposes guardrails on the use of AI to provide decision-making treatment support services, but allows the use of AI for administrative and ancillary tasks, subject to certain consent requirements. This blog post summarizes this law and addresses its potential impact on the use of agent AI by treatment providers in Illinois.
scope of law
Under this law, only licensed professionals may provide, advertise, or otherwise provide therapy or psychotherapy in Illinois.[2] “Qualified professional” includes individuals qualified to provide therapy or psychotherapy in Illinois, such as clinical psychologists, social workers, professional counselors, and marriage and family therapists.[3] The law addresses three categories of AI-supported services: (i) Administrative Support. (ii) Supplemental Support. (iii) independent therapeutic decision-making or therapeutic communication;
Permitted use of AI in treatment services
The law allows Illinois licensed professionals to use AI to provide administrative support, such as scheduling appointments, processing insurance claims, and drafting general communications related to treatment plans that do not include treatment advice.[4] Additionally, such professionals may continue to use AI to provide supplementary support such as maintaining client records including treatment notes, analyzing anonymized data to track client progress, and identifying referrals.[5] However, if a client’s session is recorded or transcribed, the law requires qualified professionals to obtain written consent from the client to use AI for supplemental support.[6] Clients or their legally authorized representatives must be informed in writing that AI will be used and the specific purpose of the AI tool or system.[7] Written consent must be affirmative and unambiguous. A general terms and conditions agreement that incorporates information about the use of AI (e.g. a general consent form for treatment) is not sufficient to establish consent under the Act.[8]
Prohibition on the use of AI in treatment services
Most importantly, the law prohibits the use of AI to: (i) make independent treatment decisions; (ii) provide therapeutic communication directly to the client; (iii) make treatment recommendations or treatment plans without review and approval by a licensed professional; or (iv) detect emotions or mental states.[9] “Therapeutic communication” is broadly defined to include “verbal, nonverbal, or written interactions that occur in a clinical or professional setting for the purpose of diagnosing, treating, or addressing an individual’s mental, emotional, or behavioral health concerns.”[10] Any person, corporation, or entity found to be in violation of the Act may be subject to civil penalties of up to $10,000 for each violation and may be subject to investigation by the Illinois Department of Financial and Professional Regulation.[11] The law also provides that therapy or psychotherapy records may not be disclosed except as required by the Mental Health and Developmental Disabilities Confidentiality Act.[12]
Impact on agent AI
Agentic AI – Autonomous AI systems that can perform a wide range of tasks, including providing test results, recognizing emotional and mental health concerns, and contacting emergency services if a user is in crisis, are being deployed in therapeutic and psychotherapy practices across the country. The law prohibits AI from making independent treatment decisions, creating challenges for providers and companies looking to use agent AI to recognize and act on users’ mental health conditions in Illinois. Providers and companies using this technology must ensure that agent AI functionality does not constitute independent therapeutic decision-making or therapeutic communication. Additionally, companies must obtain clear and affirmative written consent from clients when using these tools for supplemental support operations. Agent AI holds great promise for the future, but providers and businesses must ensure their use is within Illinois’ new restrictions.
footnote
[1] The law defines “artificial intelligence” by reference to the Illinois Human Rights Act. “A machine-based system that infers, for explicit or implicit purposes, how to generate outputs, such as predictions, content, recommendations, decisions, etc., that may affect a physical or virtual environment, from inputs it receives.” 775 Ill. Comp. Statistics 5/2-101(M).
[2] HB 1806 § 20(a).
[3] HB 1806 § 10.
[4] HB 1806 § 15(a).
[5] HB 1806 § 10.
[6] HB 1806 § 15(b).
[7] HB 1806 § 15(b).
[8] HB 1806 § 10.
[9] HB 1806 § 20(b).
[10] HB 1806 § 10.
[11] HB 1806 § 30.
[12] HB 1806 § 25.
