Evidence-First Decision Records: Reconstructing What the Agent Knew

For: risk, audit and governance leaders, and the architects who serve them. The decision: what an agent-assisted process must keep so that a decision can be reconstructed and challenged years later.

The launch note, Why Enterprise AI Needs a Deterministic Shell, argued for a Probabilistic Core that interprets and proposes inside a Deterministic Shell that owns identity, authority, state and commitment. The shell decides whether an action may happen. This note is about what it must leave behind once it has.

Fourteen months later, nothing is where it was

Illustrative scenario: a fictional composite written to make the point concrete. It does not describe a real organisation, and its figures are not reported outcomes.

Consider a home insurer that uses an agent-assisted process to triage storm claims. On 9 July 2025 it declines a claim for ceiling damage in a two-storey house. The agent’s recommendation says the staining shows long-term water entry through a worn roof, which the policy excludes as gradual deterioration, rather than sudden storm damage. A claims officer reviews the recommendation and confirms the decline.

The customer complains, the internal review upholds the decision, and in September 2026, fourteen months after the decline, the case is before an external dispute resolution scheme. The scheme asks a simple question: on what basis was this claim declined?

Everything has moved. The policy wording was reissued in March 2026. The model behind the agent has been replaced twice. The assessor reissued the site report in October 2025 with a corrected photo set. The weather feed revised its records for that week. The team can produce the agent’s explanation, a transcript and a large volume of application logs. None of it answers the question.

Explanations and logs answer different questions

The most tempting record is the agent’s own explanation. It is well written, it cites the exclusion, and it reads like a reason. But it is text the model produced about its output, not evidence of what caused the output. Chen and colleagues (2025) found that when reasoning models used a planted hint to reach an answer, the rate at which their reasoning disclosed that hint was often below 20 per cent. An explanation shows what the system said. It does not establish why it acted.

Logs fail differently. They record that a retrieval call ran, that a document identifier came back and that a model endpoint answered. They rarely record which version of the document was returned, which parts of the recommendation were facts and which were inferences, or what the claims officer actually saw when she confirmed. And if the retrieval layer resolves “policy wording” to the current version, re-running the lookup today quietly substitutes the 2026 wording into a 2025 decision.

The worst option is to re-run the case through today’s agent. That produces an answer under today’s wording and today’s model. It is a counterfactual, useful for testing, and not a reconstruction of anything.

The CAE reading: keep the basis, not the story

In the Contextual Agentic Enterprise (CAE), the author’s proposed framework for governed AI agents, the Evidence-First Decision Record is an architecture pattern: a record that keeps claims, evidence, policy decisions, approvals, versions and outcomes separately retrievable, so that a decision can be reconstructed without trusting the model’s own account of it.

Two features carry most of the weight. First, every material claim is marked by how the enterprise knows it: observed from an authoritative source, declared by a person or system, or inferred by a model. Second, evidence is referenced by version, not copied wholesale and not pointed at “latest”. The record holds identifiers and digests; the documents stay in their governed stores, which must keep old versions retrievable.

Time matters twice. Chapter 7 of the book distinguishes when something was true in the world from when the system recorded it. The July 2025 decision must be judged against the wording in force on the date of loss, and against the site report the insurer actually held on 9 July, not the corrected one it received in October.

In grouped form, a record answers four things: which decision this was and in what business context; what it rested on, meaning the inputs, versioned evidence and typed claims; how it was reached, meaning the model and agent versions, policy decisions, tool results and stated uncertainty; and who stood behind it and what followed, meaning approvals, the final state change and the later outcome. The book gives the full element list.

A reconstruction test for any consequential decision

Pick a decision at least six months old and try to answer these questions without calling a model and without opening the current version of anything.

1. Which decision, which case, which purpose? The record identifies the decision and its business context, and the Context Contract revision that shaped what the agent saw (see Context Is Not a Prompt).
2. Which evidence, by version? Each reference resolves to the exact version used, and the store still holds it.
3. Which claims were observed, declared or inferred, and by whom? No inference appears as a fact.
4. Which rules applied? The policy decision names the policy and its version, as in force for the decision’s valid time.
5. Who approved what? The approval is bound to the specific recommendation and evidence reviewed, not a Yes on a screen.
6. What changed afterwards? Later corrections, complaints and outcomes are linked to the decision, not written over it.

Any answer that begins “we would need to ask the agent” or “the current version says” is a failed test.

How decision records fail

Pointers to “latest”. The easiest defect to miss. A reference without a version reconstructs today, not then.

The explanation stored as the record. Fluent reasoning text becomes the only basis on file, and it cannot be checked.

Inferences promoted to facts. “Roof poorly maintained” is written back to the customer’s profile and shapes the next claim. An inference should stay one until a governed process promotes it.

A dump of every token. Storing full prompts and outputs by default creates privacy exposure and a haystack. The record is an assurance object, built for challenge.

Retention shorter than the challenge window. If source versions or model configurations are purged before a dispute can reach its end, the references resolve to nothing.

The pattern does not make a decision right, and it does not reveal what happened inside the model. It will not settle whether the staining was storm damage; that remains a question for assessors and, here, for the scheme. It also costs something: versioned stores, retention rules and discipline at the Commit Boundary, which is where the record should be written (see The Commit Boundary).

Who does what

Executives decide which decision classes must be reconstructable, for how long, and who answers for the record when a regulator or dispute body asks, aligning retention with the organisation’s real challenge windows.

Architects design versioned evidence stores, claim typing and bitemporal references, and write the record at the point of commitment so it cannot drift from what was done. The evidence bundle evaluated at that point, covered in The Minimum Action-Evidence Bundle in two weeks’ time, should flow straight into it.

Engineers implement and test reconstruction as a scheduled test: sample old decisions, resolve every reference, and fail the build when any points to a mutable or deleted source. They distinguish forensic replay, which rebuilds what happened, from counterfactual replay, which asks what a new version would do.

What the record says about the declined claim

The table changes the conversation at the scheme. The decline rested on one inference about staining, drawn from a photo set the assessor later corrected, and confirmed by a person who saw that inference, not the photos. That may still have been a reasonable decision. What matters is that the insurer can now show exactly what it was.

Key takeaways

• A model’s explanation records what the system said, not why it acted, and cannot serve as the audit record.
• Logs show that calls happened but rarely preserve which version of evidence was used or what was fact and what was inference.
• An Evidence-First Decision Record keeps claims, evidence, policy decisions, approvals, versions and outcomes separately retrievable, with each claim marked as observed, declared or inferred.
• Evidence must be referenced by version and judged by both the time it was true and the time the system held it.
• Re-running a case through today’s agent is a counterfactual test, never a reconstruction.

Sources and further reading

• The Contextual Agentic Enterprise, Chapter 16 §16.4 (the Evidence-First Decision Record), Chapter 7 (claims, evidence and time) and Chapter 17 (historical replay).
• Chen, Y., Benton, J., Radhakrishnan, A. et al., “Reasoning Models Don’t Always Say What They Think”, arXiv:2505.05410, 2025. https://arxiv.org/abs/2505.05410

Where the book takes this further

The book gives the full decision record, how it connects to explanation and contestability, and how forensic and counterfactual replay depend on versioned operational context. For what happens when a workflow is replayed after an effect has already reached the world, see Checkpointing Is Not Rollback, coming in three weeks.

This article is part of CAE Field Notes, a series accompanying the book, The Contextual Agentic Enterprise.