States are enacting laws to require human oversight of AI in healthcare, a response to the rapid deployment of artificial intelligence in clinical settings, KFF reported.
As of mid-October, 108 of 115 AI-related measures passed by state legislatures this year were signed into law, according to a new report from the Transparency Coalition. The most common issues addressed were human oversight of AI in health care and AI disclosure requirements.
Eight states enacted laws limiting health insurers from relying solely on AI to deny or reduce coverage, while at least five states restricted AI in mental health care, requiring licensed human professionals to provide therapy. California also passed a law ensuring clinicians retain independent judgment when using clinical AI tools, with some laws set to take effect in January 2027.
These efforts are part of a broader push to establish accountability for AI tools that increasingly influence medical decisions. Many congressional Democrats have opposed overriding state AI laws without federal regulations in place.
How State Laws Shape AI Accountability
The state-level actions reflect a common focus on the role of human judgment and accountability when AI is used in healthcare. These laws and guidance documents address different settings and establish different requirements, but they all grapple with fundamental questions about how to integrate AI into clinical practice while ensuring patient safety and maintaining professional responsibility.
States Ensure Human Oversight for AI Healthcare Decisions
- States are acting to ensure a human doctor is always in the loop when AI makes decisions about your health insurance or mental health care.
- New laws aim to make it clear who is responsible if an AI tool gives bad medical advice, shifting the focus from the machine to the clinician using it.
- While some laws are already active, others will take effect in 2027, signaling a continued, evolving effort to regulate this technology.
Medical boards and professional organizations are also developing guidance for the use of AI in clinical practice. Unlike state laws, these recommendations are voluntary and do not establish binding requirements. The AMA says these questions will guide its work on liability, licensure, and regulation.
Regulatory Approaches to AI in Healthcare Vary Globally
The regulatory push in the United States is part of a global conversation about AI in medicine. While this article focuses on the American landscape, similar concerns are being addressed by international bodies.
| Region | Regulatory Approach to AI in Healthcare | Key Principle |
|---|---|---|
| United States (State Level) | Specific laws on insurance denials and mental health AI use. | Legal accountability and human judgment. |
| United States (Federal Level) | Potential national framework, preemption debates. | Uniform standards vs. state flexibility. |
| European Union | AI Act, focusing on high-risk systems. | Risk-based oversight and conformity assessments. |
| Global (WHO) | Guidelines on ethics and governance of AI for health. | Transparency, validation, and equity. |
The Role of Scientific Publishing in Setting Standards
Academic publishers have also taken a stand on the use of AI in research and publication. This is a critical point of data integrity, as the foundation of medical research relies on the verifiability of its sources.
These efforts, while operating in different spheres—law, professional practice, and publishing—address a common set of questions about the role of human judgment and accountability.
Patients Can Question AI Systems Used in Their Care
This article addresses the regulatory and professional frameworks governing AI in healthcare; it does not constitute medical advice. There are no “contraindications” for AI oversight laws themselves. However, for patients, the key takeaway is about awareness and advocacy.
If you have concerns about an AI system being used in your care, you have the right to ask your clinician how the tool is being applied and what their role is in overseeing its recommendations.
Consult your doctor or a healthcare professional if you experience any negative health outcomes or have persistent doubts about the care you are receiving. You should also raise concerns with your state medical board if you believe a clinician is improperly delegating their professional responsibilities to an AI tool without the required oversight.
The focus on human oversight is directly responsive to public attitudes.
Looking ahead, the KFF will continue to monitor how state and federal officials approach AI regulation. The interplay between state innovation, federal preemption, and professional guidance will determine how AI develops in medicine, with the ultimate goal of ensuring these powerful tools enhance, rather than undermine, patient care.