PHI Risk in AI-Powered Insurance Claims Processing

AI claims processing does not touch protected health information by accident. The workflows that create the biggest speed gains, conversational first notice of loss, multimodal damage intake, and automated subrogation, are the same ones that produce unstructured PHI in volume. The exposure is structural.

How AI claims workflows generate PHI at scale, not just in isolated touchpoints

The claims lifecycle now has five AI-accelerated layers where PHI enters the pipeline. Conversational FNOL intake captures claimant narratives. Multimodal photo and video damage assessment processes evidence. Automated subrogation research reasons over third-party records. Fraud detection reads behavioral and audio signals. Claims-status conversation agents handle follow-up. Each layer generates a different PHI profile, and each operates on raw input, not pre-filtered fields.

The common thread across all five is a change in how data reaches models. LLMs now ingest free-form claimant narratives, medical descriptions, injury accounts, and third-party records as their native input. This is not structured data passed through upstream filters. PHI arrives at the model in its most uncontrolled form, embedded in sentences rather than separated into governed fields. The volume scales with every workflow an insurer adds.

What conversational FNOL actually captures and why it is the highest-PHI-density touchpoint

Conversational FNOL is where PHI enters the AI pipeline in its rawest and most legally sensitive form. A policyholder narrating a loss, a kitchen fire, a collision, a hospitalized child, does not follow a structured template. The interaction produces a free-form narrative. Injury descriptions, diagnoses, names of treating physicians, medication references, and details about dependent family members may all appear in a single call.

The extraction process makes this more consequential. LLMs pull structured data from that narrative on the fly, so the full unredacted text passes through the model prompt before any extraction occurs. Every PHI element the policyholder mentioned is in the context window at once. No upstream filter removed it, because there is no upstream filter for conversational input.

Synthetic voice fraud adds a second exposure vector at the same layer. The same FNOL conversation channel that captures health and injury narratives is the attack surface for cloned policyholder voices. Synthetic voice attacks against insurers rose sharply in 2024. The channel handles both legitimate PHI and fraudulent audio, which means any control applied to it must contend with both

How multimodal damage intake and automated subrogation extend PHI exposure beyond the first call

PHI generated at FNOL does not stay in one system. Photo and video damage models, automated subrogation engines, and agentic decisioning layers each pull, transform, and route claim data, including medical and injury information, through additional model calls. A single claim now passes through several AI workflows, and the PHI travels with it.

Multimodal photo damage assessment is the most mature AI claims category. It is primarily a property and auto tool, but injury-adjacent claims change its exposure profile. In a slip-and-fall or a vehicle accident with bodily injury, images submitted with medical invoices or hospital discharge summaries can carry PHI in document metadata or visible text. The model ingests what the claimant uploaded, not just what the workflow intended to collect.

The scale of this movement is production reality, not an edge case. CCC Intelligent Solutions processes billions of transactions daily across hundreds of insurers and tens of thousands of repair facilities as of its Q1 2026 earnings. Each transaction is a potential PHI-adjacent document flowing through an AI pipeline. Automated subrogation extends the exposure further. This AI identifies, scores, and pursues third-party recovery, which requires it to ingest and reason over third-party medical records, police reports, and liability correspondence. Those documents may contain PHI about parties who never consented to have their data processed by the insurer's AI vendor.

Agentic workflows compound the problem. A single claim may chain multiple model calls across FNOL intake, damage assessment, coverage determination, and subrogation evaluation. PHI sits in the context window at each step and may be logged at each hop. Aviva deployed more than 80 AI models for motor claims, cutting complex liability assessment by 23 days and improving routing accuracy, and that is the clearest production evidence that PHI travels through many model layers per claim, not just one.

Why standard PII detection fails to catch the PHI that insurance AI workflows actually produce

The PII detection tooling most teams deploy by default uses regex patterns covering SSNs, phone numbers, and email addresses. That approach is built for structured data. It cannot recognize the unstructured, clinically inflected PHI that FNOL narratives and medical document ingestion produce.

Pattern-based detectors find values that follow a predictable format: a nine-digit SSN, a ten-digit phone number, a sixteen-digit card number. They do not find "the patient was diagnosed with lumbar disc herniation at St. Francis Hospital" or "my son takes Adderall for ADHD." Both phrases are definitionally PHI under HIPAA when associated with an identifiable individual, but neither matches a pattern. Recognizing clinical PHI requires a semantic entity recognizer, a tool trained to identify diagnoses, medications, body parts, treatment references, and provider names as PHI. Concentrate includes semantic PHI redaction at the gateway layer, applied to each request before it reaches any model provider, so clinical context that regex rules would overlook is masked rather than passed through.

The gap matters most at the FNOL layer, where narrative input is least predictable. It applies equally to any workflow that ingests medical invoices, discharge summaries, or handwritten claim notes through a document-parsing model. And there is a second failure mode: redaction applied only to the prompt, not the response. A model's generated summary of a claim, "claimant sustained a fractured tibia requiring surgical fixation," may itself be PHI if linked to an identifiable person. That summary is often what gets stored in downstream systems.

What HIPAA and state AI regulations require from an AI claims pipeline specifically

HIPAA compliance for an AI claims workflow is not a single checkbox. It is six concurrent requirements that must hold across every model call that touches PHI, and each one implies a specific infrastructure control. The six are: a signed Business Associate Agreement with every model provider that receives PHI, data retention limits and defined deletion timelines by those providers, encryption in transit and at rest, access controls plus audit logging, PHI minimization before data reaches the model, and breach notification readiness.

The BAA requirement is the first chokepoint. A carrier cannot send PHI to a model provider, even temporarily, even in a prompt, without a signed BAA in place. Most general-purpose model provider contracts do not include a BAA by default. Audit logging is where most enterprise AI deployments fail a separate requirement. AI systems frequently access regulated data under a shared service account or API key, so no log records which individual directed the access. HIPAA's unique user identification standard at §164.312(a)(2)(i) and audit controls standard at §164.312(b) together require that attribution.

RBAC is not optional either. HIPAA requires that access to PHI be limited to the minimum necessary for the authorized purpose. In an AI context, that means different teams, applications, and agents should be granted access only to the model calls their role requires. The litigation context sharpens these stakes. The UnitedHealth nH Predict case, where a federal court allowed a case alleging a high error rate in AI claims decisions to proceed in early 2025, and the Cigna PXDX ProPublica investigation both illustrate that AI handling PHI without documented controls attracts regulatory and litigation exposure, even when the primary complaint is denial accuracy rather than a data breach.

Where PHI governance must be enforced in an AI claims architecture and what that enforcement looks like

Enforcing PHI governance across a multi-model, multi-vendor claims pipeline requires a single enforcement boundary that every model call passes through. Per-integration controls, applied inconsistently, guarantee that one missed integration becomes a HIPAA violation. An AI gateway sits between the claims application and every model provider, running guardrails on each request before it reaches the provider and on each response before it returns. That makes it the one place where PHI redaction, BAA-covered routing, access control, and audit logging can be applied consistently.

At the redaction layer, effective PHI governance requires semantic entity recognition, not just regex patterns. Clinical language in FNOL narratives and medical document summaries must be caught before it reaches the model provider. At the routing layer, the gateway must know which providers have signed BAAs and route PHI-bearing requests only to BAA-covered endpoints. That policy cannot be maintained reliably in application code across a pipeline with many model calls.

At the audit layer, the gateway must log every request with individual caller attribution, not a service account. HIPAA's unique-user-identification requirement can only be met that way, and a market conduct examination or litigation discovery request can then be answered with a complete, attributable record. Zero data retention must be enforced by policy at the gateway, not assumed from provider contracts. A gateway should be configured to route only to providers whose ZDR terms are confirmed, and that configuration should be auditable. Because PHI flows through multiple model layers per claim rather than stopping after intake, teams need visibility into what data reaches each provider and the ability to enforce consistent redaction policies across the entire pipeline. Concentrate provides that visibility and enforcement at a single chokepoint, before any model ingests the data.

The governance controls an insurer should map across its AI claims pipeline before adding more models

Before expanding AI claims capabilities, an insurer needs a clear map of where PHI enters, moves, and is governed across its existing pipeline. Adding models to an ungoverned architecture scales the exposure, not just the speed. The mapping exercise has five steps.

Step one: inventory every AI touchpoint in the claims lifecycle against the five workflow layers described above, FNOL intake, damage assessment, subrogation, fraud detection, and status communication, and identify which touchpoints currently receive unstructured claimant input or ingest third-party medical records. Step two: for each touchpoint, confirm whether the model provider has a signed BAA in place, whether ZDR is contractually enforced, and whether the redaction applied is semantic or purely pattern-based. Step three: audit the access control and logging posture. Are model calls attributed to individual users or to shared service accounts? Can the organization produce a complete audit trail for a specific claim, showing every model call, the identity of the caller, and what was redacted?

Step four: evaluate whether enforcement is applied at a single gateway layer or distributed across individual integrations. If distributed, treat every integration without a gateway as an uncontrolled exposure point. Step five: confirm that RBAC is configured so that different teams and agents can access only the model calls their role requires, and that budget and rate-limit controls are in place by team, project, and model. Both matter: role restrictions preserve the minimum-necessary standard, while budget controls satisfy the NAIC's cost-allocation requirement and catch runaway spend from agentic workflows before it surfaces on an invoice.

The NAIC's AI governance framework and the state-level requirements described above are the external audit criteria against which this mapping will be tested. Building the map now is what makes a market conduct examination answerable. Governance is not the constraint on AI adoption speed. Ungoverned PHI exposure is.