AI Is Quietly Influencing What Care Gets Approved – Should Patients Be Told?

When a health insurer denies a claim or declines to authorise a procedure, the decision arrives as a letter or a notification. It cites a policy reason, names a human reviewer and doesn’t tend to mention whether an algorithm made the call first, shaped the recommendation, or determined the outcome entirely. In a growing number of cases, that’s exactly what’s happening.

AI is now being used across the health insurance claims cycle – prior authorisation, retrospective review concurrent review – and a May 2026 brief from KFF reported that 84% of insurers responding to an NAIC survey across 16 US states said they were already using AI or machine learning for tasks including utilisation management and prior authorisation.

With the technology now implemented, the focus has shifted to patient rights and institutional accountability in the event of failure.

How AI is Reshaping Claims Review

Automating claims processing offers a clear advantage for health insurers, as AI can efficiently handle the high volume of standardised, repetitive requests inherent in the industry. AI can triage that volume, flag inconsistencies, extract relevant data and flag cases that need human attention faster than a manual review process can manage. Proponents argue this reduces administrative cost, speeds up approvals for clear-cut cases and frees human reviewers for the complex ones.

As the Stanford HAI policy team points out, AI thrives in the very type of high-volume, repetitive environment found in claims processing. For first-pass sorting, document extraction and initial eligibility checks, the efficiency gains are measurable. The KFF brief supports this reading: the technology is doing useful work in claims processing, and the insurers using it aren’t doing so without reason.

The situation becomes more complex with adverse determinations. When AI influences decisions to deny coverage or flag claims for rejection, tracing the resulting chain of accountability becomes significantly more difficult. Patients who challenge a denial have historically struggled to get clear information about why a decision was made. Add an algorithmic layer to that process and the opacity problem compounds.

Several class actions have already been brought in the US alleging improper denials tied to algorithmic decision-making, and the courts are only beginning to work through what liability looks like in this context.

The Oversight Problem

In June 2026, MACPAC – the Medicaid and CHIP Payment and Access Commission – published a report recommending that all adverse prior authorisation determinations in Medicaid be reviewed by a person with appropriate clinical expertise, and that AI and automation tools may not make those decisions alone. Separately, it recommended that states require Medicaid health plans to disclose their use of automation to patients, with oversight, testing and evaluation built in.

State legislatures across the US are actively considering similar bills. The path for future regulation is taking shape, centred on three pillars: requiring transparency when AI is used, ensuring a human evaluates every denial and guaranteeing that patients can effectively challenge automated decisions.

The greater challenge lies in defining the practical application of ‘meaningful human oversight’. A clear distinction exists between substantive human oversight – in which a clinician conducts an independent assessment – and performative oversight, where a reviewer simply validates an AI recommendation without properly interrogating it, effectively rubber-stamping an automated decision.

The transparency debate and the oversight debate are different, but they point at the same underlying problem: the accountability infrastructure hasn’t caught up with the technology that’s already inside the system.

The Future Of Compliance For Health Tech Builders

For founders building in claims automation, prior authorisation tooling or any adjacent part of the health insurance workflow, the regulatory direction now being set by MACPAC and state legislatures is the product roadmap. Disclosure-by-default, human-in-the-loop architecture for adverse determinations and audit trails that can demonstrate what role automation played in a given decision aren’t features that can be bolted on later.

The companies that will be best positioned as oversight requirements arrive would likely be those building with the assumption that regulators will eventually want to see exactly what their system did at each step of a decision, and that patients will eventually have the right to know. That’s a higher bar than the current environment requires – it’s probably the right bar to be building to.