Use cases Healthcare

Use case · Illustrative

Care-coordination agent

A coordination agent updates patient records within a granted scope. A valid token and a lagging view are enough to change the wrong thing, correctly.

01The workflow

An AI agent keeps patient records and follow-ups current: it updates medication lists, books referrals, and closes care tasks on a clinician’s approval.

02The rule that must hold

change.patient == grant.patient  ∧  change.kind ∈ grant.scope  ∧  change.based_on == record.version

A record changes only for the patient and the kind of change that was granted, and only against the record as it currently is.

03How it breaks

  1. A clinician approves a medication change for one patient; the agent receives a token scoped to that update.
  2. The agent reads the patient summary, a view that has not yet applied a change made minutes earlier in the record system.
  3. It writes the medication list back from that summary, overwriting the newer entry. The token was valid and the write was accepted.
  4. A follow-up request for a different kind of change arrives while the same token is still valid, and it is accepted too.
  5. Every call was authenticated and in scope by its token. The record is now wrong, and the approval did more than it was granted for.

04Why monitoring misses it

  • Task successThe update completed and the agent reported success.Healthy
  • Access logsA valid, unexpired token was presented for every call.Healthy
  • Record systemThe write was accepted; the record is well-formed.Healthy
  • The ruleA record changes only for the patient and the kind of change that was granted, and only against the record as it currently is.Broken

05Reproduced in the lab

Each failure mode below is reproduced deterministically in the public Agent Control Lab and can be run by its id.

06What we would engineer

07What the diagnostic delivers

  • The rules this workflow must keep, written down and classified.
  • For each rule, the system that can enforce it, or the gap where none can.
  • Priority failure modes reproduced where feasible.
  • An ordered hardening plan.

Discuss this workflow See a sample assessment (PDF)

Boundary Illustrative engagement, not a client engagement or a measured production result. The failure modes listed are reproduced in the public lab.

Real systems. Higher standards.

Bring us the AI system that has to work in the real world.

Discuss your production AI system