Checkpointing Is Not Rollback: The Dangerous Myth of Agent Time Travel
Restoring an agent's workflow state does not restore the world. Safe recovery needs an effect journal, idempotency keys and compensation governed as a new action.
Written as at 9 August 2026.
reliabilityagentic-airecoveryplatform-engineering
For: engineers and platform leads who run agent workflows in production. The decision: what your runtime must know before it repeats any step that touches the outside world.
New to the series? The Deterministic Shell is the deterministic layer around a model that decides whether its proposals may become real, and the Commit Boundary is the point where a proposal becomes an effect someone else can see. This note is about what happens when a workflow crosses that boundary twice.
A replay that paid 212 people twice
Illustrative scenario: a fictional composite written to make the point concrete. It does not describe a real organisation, and its figures are not reported outcomes.
Imagine a hospital network implementing a new enterprise agreement. Back pay is owed to 212 nursing staff, averaging $1,640 each: $347,680 in total. A payroll agent calculates each amount, the payroll manager approves the batch, and the agent submits it to the payments provider.
The provider’s API takes longer than usual and the call times out after 30 seconds. The orchestrator marks the step as failed. The next morning an engineer, seeing a failed run, uses the framework’s checkpoint feature to replay from the last good state: “batch approved, ready to submit”. The approval is part of that saved state, so the workflow submits again.
The provider had accepted the first batch. It accepts the second. Every one of the 212 staff is paid twice, and $347,680 that nobody intended to pay has left the organisation.
Checkpoints restore the workflow, not the world
Durable execution is now an ordinary feature of agent frameworks. Workflows save state after each meaningful step so a crash doesn’t mean starting again. That is valuable. The trouble is the mental model that comes with it: that returning to an earlier state is a form of undo.
At least one widely used framework, LangGraph, calls this “time travel”. To its credit, its “Use time-travel” documentation says plainly that replaying from a checkpoint re-executes the later steps, and that API requests fire again. The warning is there. It is easy to read past, because in development the API is usually a stub.
The deeper problem is the timeout. A timeout is not a failure. It is an unknown outcome. The orchestrator knew only that it had stopped waiting, and it recorded that as “the payment did not happen”.
Four recovery operations, none of which is undo
The book separates four operations that are routinely blurred. In my own summary:
- Resume picks up an interrupted run from its latest valid state and carries on.
- Retry repeats a step that failed, and is safe only when there is evidence the earlier attempt did not commit.
- Replay runs the workflow logic again from an earlier checkpoint, to recover, diagnose or reconstruct what happened.
- Fork starts a new branch from an earlier state, leaving the original history intact.
None of them reverses an effect that has already reached a bank, a ledger or someone’s inbox. Three mechanisms make recovery safe instead.
An effect journal records the link between each workflow decision and the transaction the external system created: here, the provider’s batch reference against the approved batch. An idempotency key gives each effect a stable identity, such as one key per employee per back-pay period, so that a repeated request returns the earlier result instead of paying again. And when an effect must be reversed, compensation is a new action, with its own authority, evidence, policy check and decision record.
Compensation comes from sagas (Garcia-Molina and Salem, 1987): a long-running transaction is split into steps, each paired with a compensating step that semantically undoes it when a global rollback isn’t available. Semantic undo is the point. Recovering 212 overpayments is not a database operation. It is a payroll decision about people’s pay, with its own approver, notices to staff and rules about how recovery may happen.
One more rule closes the loop. Replay must re-check the Minimum Action-Evidence Bundle, a CAE design construct for the smallest verifiable set of information needed to decide one consequential action. The approval inside the checkpoint was valid for one submission of one batch, yesterday. It is evidence of a past decision, not permission for a new one.
Ask the journal before you touch the world
Before any step that could create an external effect, the runtime follows a fixed sequence.
- Look up the effect journal using the step’s idempotency key.
- If the effect is recorded as committed, return the recorded result. Do not execute.
- If the outcome is unknown (a timeout, a crash between call and acknowledgement), query the target system by key. Until it answers, treat the step as unknown, not failed, and do not retry.
- If the target confirms nothing was committed, retry with the same key, after re-validating the approval and evidence against current conditions.
- If the effect committed and was wrong, raise compensation as a new action through the action gateway, never as an edit to history.
- For diagnostic replay, disconnect the effect adapters. A replay to understand what happened should not be able to make anything happen.
Applied to the hospital: at step 1 the journal shows the batch submitted with no acknowledgement, so step 3 applies. The provider, asked by key, reports batch accepted. The replay ends there, and nobody is paid twice.
How recovery still goes wrong
Fresh keys on every attempt. If the key is generated per call rather than per effect, the provider sees 212 new payments. The key must be derived from the business operation.
The journal written too late. Recording the effect only after the provider responds leaves a gap exactly where the timeout occurs. Write the intent first, then the outcome.
Targets without idempotency. Many legacy payment and HR systems won’t honour a key. The adapter must then check for the transaction itself before repeating anything.
Approval restored as valid. Checkpointed approvals, permits and policy decisions look current to the workflow. They must be re-evaluated at the new execution time.
Compensation as deletion. Removing the duplicate batch from the workflow’s records hides the event without recovering a cent, and destroys the evidence. The original stays; the reversal is recorded beside it.
Forks with live adapters. Two branches explored from one checkpoint can both pay.
The limits are real. None of this makes the back-pay calculation correct. Some effects have no true compensation: a message sent to a regulator cannot be unsent. And compensation can itself be partial, if some of the 212 have since left the organisation.
Responsibilities by role
Executives decide that “we can roll it back” is never accepted for an agent that moves money or changes records, and ask instead how the organisation would compensate, and who would approve it.
Architects design the separation: the framework keeps computational state, the effect journal keeps what reached the world, the gateway mediates the boundary and the Evidence-First Decision Record keeps what was decided. Chapter 17 of the book separates forensic replay, which reconstructs a past run, from counterfactual replay, which tests a new version against past cases. In CAE, neither should ever run with live effect adapters.
Engineers implement and test the failure paths deliberately: inject a timeout after the provider commits, replay from every checkpoint that precedes an effect, and confirm that each attempt ends in “already committed” rather than a second payment.
What each recovery operation can and cannot do
| Operation | What it restores | What it cannot do | Check before any external effect |
|---|---|---|---|
| Resume | The run’s latest valid state | Tell whether the last effect committed | Journal lookup for the in-flight step |
| Retry | Another attempt at one failed step | Distinguish a failure from a timeout | Target confirms no commit; same key |
| Replay | Workflow logic from an earlier checkpoint | Reverse anything since that checkpoint | Re-validate approval and evidence; adapters off for diagnosis |
| Fork | A new branch, history kept | Stop both branches acting | Adapters off, or a separate key space and approval |
| Compensation | The business position, semantically | Erase the original event | Its own authority, evidence and record |
Key takeaways
- Restoring an agent’s workflow state does not restore the external world, and a replay can repeat any effect after the checkpoint.
- A timeout is an unknown outcome, and a runtime that records it as a failure will eventually duplicate a payment.
- Resume, retry, replay and fork are distinct operations, and none of them is undo.
- An effect journal and business-derived idempotency keys let the runtime ask whether an effect already happened before it acts again.
- Compensation, in the sense of sagas (Garcia-Molina and Salem), is a new governed action with its own authority and record.
Sources and further reading
- The Contextual Agentic Enterprise, Chapter 3 (§3.6.5) and Chapter 17.
- Garcia-Molina, H. and Salem, K. (1987) “Sagas”, ACM SIGMOD Record 16(3). doi.org/10.1145/38714.38742
- LangChain (n.d.) “Use time-travel”, LangGraph documentation, accessed 22 September 2026.
What the book adds
Chapter 3 sets out how replay, the evidence bundle, the action gateway and compensation fit together at the Commit Boundary, and Chapter 17 covers forensic and counterfactual replay, reliability patterns and failure injection in depth. For the earlier pieces this note depends on, see the Governance-to-Action Gap and the Minimum Action-Evidence Bundle.
This article is part of CAE Field Notes, a series accompanying the book, The Contextual Agentic Enterprise.
Related
The Minimum Action-Evidence Bundle: What Must Accompany a Consequential Action
Evidence for an agent's action is neither a log of everything nor a fixed form. It is the smallest verifiable set that lets a control decide, and approval must bind to it.
The Commit Boundary: Where AI Recommendations Become Enterprise Consequences
Controls placed at the start of an agent conversation check the wrong thing. The decisive check evaluates the exact action about to become real, at the moment it becomes real.
Evidence-First Decision Records: Reconstructing What the Agent Knew
A model's explanation is not an audit record, and logs are not a reconstruction. An Evidence-First Decision Record keeps claims, evidence and versions retrievable long after everything has changed.
Closing the Governance-to-Action Gap with Runtime Mediation
Policies, approval screens and audit logs describe governance. Only a control on the execution path enforces it, and every consequential action must pass through one.