"in the future.” 
                                                                                     Satya Nadella, CEO, Microsoft

 

I am often asked to provide guidance on governance of artificial intelligence to various C-suites, board of directors, and health systems. Along with regulation, ethics, safety, cybersecurity, and legal issues, governance of AI in healthcare seems to be peaking as a hot mini topic in healthcare AI. Here are some of my thoughts:

Healthcare has finally noticed that artificial intelligence is here, and the regulators have noticed that healthcare has noticed. What began as polite academic conversation about model cards and fairness metrics is now a regulatory thicket spanning Washington, Brussels, and the C-suite of every health system that signed a contract with a vendor whose pitch deck contained the word "AI." Governance is no longer optional. The question is whether we govern with insight or with theater.

 

The landscape is broader than any one list captures. Beyond CHAI, ONC, DOJ, and the EU AI Act sit the FDA, the Joint Commission, ISO/IEC 42001, NIST, the Office for Civil Rights, the FTC, the WHO, and an accelerating tide of state laws. What follows compresses that constellation into the six forces that actually matter for an operating health system in 2026 — five regulatory or enforcement bodies plus the one international standard that ties them together.

 
The Top Forces in Healthcare AI Governance

1. The FDA
The senior regulator. By early 2026, the FDA had authorized over 1,350 AI-enabled medical devices, roughly double the 2022 count. Its August 2025 final guidance on Predetermined Change Control Plans (PCCPs) lets manufacturers ship algorithmic updates without filing a new submission, provided the changes fall within a pre-cleared envelope. The January 2025 draft Total Product Life Cycle guidance is the FDA's first comprehensive AI playbook: model description, data lineage, performance against claims, bias mitigation, human-AI workflow, and postmarket monitoring in one document. The Quality Management System Regulation (QMSR), aligning the U.S. with ISO 13485, takes effect February 2026. Translation: if your device learns, you must say in advance how it will learn and prove afterward that the learning did not break it.

 

2. ONC/ASTP HTI-1
ONC's HTI-1 final rule was the federal government's first swing at AI inside the EHR rather than inside a discrete device. By replacing the old clinical decision support criterion with the new Decision Support Intervention (DSI) criterion, ONC required certified vendors to expose thirty-one source attributes for every Predictive DSI — the regulatory cousin of the model card. The DSI criterion took effect end of 2024; USCDI v3 becomes baseline in January 2026. The proposed HTI-5 rule would scale back some disclosures, but until finalized, transparency is the floor: every certified EHR in the country must describe how its embedded algorithms were trained, validated, and maintained.

 

3. CHAI and the Joint Commission
CHAI began as industry self-regulation before government did it for us, and its Responsible AI Guide and draft model cards now form the operational vocabulary at most academic medical centers. The proposed assurance lab network has stalled, but the September 2025 joint guidance with The Joint Commission has filled the vacuum with something more durable: an accreditation-linked framework built on FAVES principles — fair, appropriate, valid, effective, safe. A voluntary Joint Commission AI certification, derived from CHAI playbooks, launches late 2026 and will reach 23,000 accredited organizations. Industry consensus has become, in effect, soft regulation.

 
4. The EU AI Act
Across the Atlantic, the EU AI Act classifies essentially all clinical AI as high-risk by design. AI literacy obligations and prohibitions on unacceptable-risk systems are already in force. The high-risk regime was set to take full effect August 2026, but as of 7 May 2026 a Digital Omnibus political agreement staggered the deadlines: standalone Annex III systems get a sixteen-month extension, AI embedded in CE-marked medical devices a twelve-month extension. Manufacturers still face dual conformity assessment against the AI Act and MDR/IVDR, plus Article 86's right to explanation for adversely affected patients. American health systems that touch European data or deploy European-made devices cannot pretend this is someone else's problem.

 

5. The DOJ and OCR Enforcement Axis
The DOJ is the sleeper enforcer that AI leaders ignore at their peril. FY 2025 set a record at $6.8 billion in False Claims Act recoveries, 84 percent in healthcare. The DOJ-HHS FCA Working Group has named AI-driven EHR manipulation a priority target, and the $556 million Kaiser Permanente settlement in January 2026 over Medicare Advantage coding involved exactly the kind of algorithmic prompting that lives inside modern EHRs. Alongside DOJ, the HHS Office for Civil Rights enforces HIPAA and Section 1557 nondiscrimination — meaning an algorithm that systematically underserves a protected group is now a civil rights matter. If an algorithm nudges a clinician toward a billable diagnosis the patient does not actually have, the claim is false, and per-claim penalties compound.

 

6. ISO/IEC 42001
If the other five forces are regulators and enforcers, ISO/IEC 42001 is the operating standard that makes the whole thing auditable. Published December 2023, it is the world's first certifiable AI management system standard — the AI equivalent of ISO 27001 for information security or ISO 13485 for medical devices. It is industry-agnostic and jurisdiction-agnostic, which is precisely its power: a single management framework that satisfies the EU AI Act, aligns with NIST's RMF, supports HIPAA expectations, and produces the documentation the DOJ would want to see. Microsoft achieved certification in 2024; healthcare adoption is accelerating as 42001 becomes a procurement prerequisite for AI vendors selling into health systems. For a hospital, the practical implication is binary: either build an AI Management System (AIMS) that maps cleanly to 42001 clauses, or be prepared to assemble that documentation under pressure when a regulator, accreditor, or plaintiff's lawyer asks for it.

 

Behind these six sit the supporting cast: NIST's AI Risk Management Framework (the technical baseline), the WHO's ethics guidance, the FTC for deceptive AI marketing, and state laws from California's AB 3030 to Colorado's algorithmic discrimination act. Six forces lead; ten more shape the field.