Aggregate invariants in action-taking agents

01The question

Can an action-taking agent leave the business in a state nobody authorized while every local control holds? The invariant under test:

sum(refunds) + sum(credits)  ≤  authorized_compensation

Refunds and credits live in separate databases with no shared transaction. A CHECK constraint cannot read another database, so neither service can see the sum it is part of. The agent reads a CRM projection, a reporting view it does not own and which lags, because that is what agents are usually given.

02Method

• Deterministic schedules. Each interleaving is scripted, so every verdict is attributable to one variable. A verdict is void if the schedule did not actually execute.
• An oracle that proves itself first. Before every schedule it must catch a planted violation and pass a planted safe state. It writes its own SQL, imports no service code (checked on the syntax tree) and connects with credentials that cannot write.
• Negative controls are the point. Five schedules must come back clean. A harness that only produces violations proves nothing.
• Determinism is measured. Across about 700 replays, several hundred under a process writing to and truncating the same databases, the release order never varied.

03Results

Twelve schedules; three end with the business state wrong, and two of those are invisible to ordinary monitoring. The table is generated by the run that asserts it (RESULTS.md ↗).

A monitoring “finding” is not detection of the breach. The reconciler looks for generic anomalies and is deliberately not told the ceiling. P1, P2, P0 and P3 report lag only because they have no projector, so their events are never consumed.

04The finding that matters

Everyone expects the race: two agents read, both see $0, both act. That is P2, and sequencing the agents fixes it.

S1 is not a race. Agent B starts reading strictly after agent A has committed and been acknowledged. Nothing overlaps. It is still wrong, because the view it read had not applied A’s event. S1C, the same schedule with the projection caught up, is clean: the only variable is whether the view had caught up. S3 closes the remaining escape route: a partially caught-up view still breaches. Freshness narrows the window; it cannot close it.

05Why monitoring misses it

None of these signals is broken; each answers its own question correctly, and none is asked whether the business state is right. Time to detect is not long but undefined: the reconciler never reports the breach because nothing it checks is wrong. That null result is calibrated against a planted defect the reconciler does catch.

06With a real model

The deterministic arm is the stronger evidence for the structural claim, because no failure can be blamed on a prompt. The model arms answer a different question. Same workload, a $1,000 ceiling, a $600 refund and a $500 credit launched concurrently, deepseek-chat at temperature 0, five runs each (LLM-ARMS.md ↗):

In every arm C run, both agents read the compensation first, saw the pre-write state, and acted anyway. The model was diligent and wrong for the same reason the deterministic agent is. Arm D is the result worth having: the fix does not depend on cognition.

07Exposure and capacity

Without a scripted interleaving, exposure depends on how far apart the agents arrive. Ten runs per point (MODE-B.md ↗):

The cliff sits between 75 and 125 ms: the duration of the agent’s own read-check-write sequence. At 100 ms the window was open in every run while only 20% violated, so an open window is necessary but not sufficient, which is why a bare frequency cannot be interpreted.

The authority under contention: a $1,000 ceiling, every agent requesting $100, so exactly ten can be granted (CAPACITY.md ↗):

08What fixes it, and what it costs

Not diligence and not freshness: an authority that can see every input to the invariant at decision time, granting before the action rather than validating after it, with the check and the grant as one atomic step. P0 and S1H apply that one reservation to both failure modes and both stay clean; S1H holds the staleness constant, which locates the fault in the interface the agent was given, not in the agent.

The cost, stated as a cost: a serialisation point, and latency that grows linearly with contention on the same case. Contention is per case; agents working on different cases do not queue. Most invariants are local, and an authority bought for a local invariant is a serialisation point bought for nothing.

09Outside the lab

Each source was read at its primary record, and each states what it does not support (PRIOR-INCIDENTS.md ↗). The harm is documented; the mechanisms differ.

• ACIDRain (SIGMOD 2017) ↗

  Supports12 eCommerce platforms deployed on over 2M websites; 22 verified attacks that over-spend gift cards and corrupt inventory, 17 of them even under the strongest transactional guarantees those databases offer.

  Does not supportThe applications share one database, so a correctly scoped transaction exists in principle. Same class of harm, reached by an easier road.

• Twilio billing incident (2013) ↗

  SupportsEach recharge was a valid response to the state the biller could see, and wrong against the separate store holding the actual account status. 1.4% of customers affected; found through customers, not monitoring.

  Does not supportTwilio’s view was wrong through data loss, not propagation lag. The structure transfers; the mechanism does not.

• US Dept. of Education NSLDS (2024) ↗

  SupportsOnce corrected, aggregate loan totals pushed about 36,786 borrowers past their aggregate limits after institutions had already disbursed on the wrong figures.

  Does not supportA component was wrong: the aggregation itself had a defect. Adjacent evidence about aggregate ceilings, not this mechanism.

What is missing publicly is an instance where no single transaction could have enforced the invariant because the data was never in one place. That is the case this lab constructs deterministically, because finding it in the wild requires access to systems that do not publish their postmortems.

10What this does not claim

• Read-then-write across uncoordinated systems is textbook and is not presented as a discovery. What is measured is the combination: the failure survives authentication, authorization, idempotency, strict sequencing and competent monitoring at once.
• No model is in the deterministic path, deliberately. It says nothing about model behaviour; the model arms are separate.
• Not a production frequency. One machine, local services, no network latency, synthetic data.
• One intermittency in the multi-schedule report generator (ADR-015) is open, narrowed but unexplained.
• Replay inside a token’s validity window is not prevented. It is named in the threat model ↗ rather than implied away.

11Applying this to your system

1. Write your invariants down and classify them as local, cross-service, eventual, hard or bounded-time (taxonomy ↗). Most teams never have.
2. Name the authority for each cross-service hard invariant. If the answer is “each service checks its own part”, the invariant is unenforced.
3. Measure your exposure window: your agent’s read-check-write duration and how close together agents arrive. Both are measurable in production.
4. Add coordination only where step 2 found a gap.
5. Check invariants separately from operational health, or the failure stays invisible exactly as it does here.

Step 1 is free and frequently ends the discussion: either every invariant is local, or one is not and it has no owner.