Ambiguity, Idempotency, and Recovery

Unknown is a valid result

A timeout does not reveal whether an operation failed before execution, succeeded before the connection broke, or remains in progress. Treating every timeout as failure invites duplicate effects; treating it as success invents closure.

Unknown resultLook up stable keyCompare the worldAccept · retry · escalate

Represent ambiguity directly. Preserve the attempt identifier, stable operation key, request fingerprint, authority snapshot, timestamps, transport observations, provider identifiers, and required proof. Recovery can then learn what happened instead of replaying from a blank state.

Give one logical effect one stable identity

An idempotency key identifies a logical operation across retries. It should be created before the first attempt and reused only for the same normalized request, principal, target, and effect type. A provider can use it to return the prior result or reject conflicting reuse.

Idempotency is a scoped contract, not a universal property. Providers differ in supported operations, retention windows, parameter comparison, account scope, and replay behavior. The harness must record those limits and avoid promising exactly-once execution.

A random key on every retry defeats deduplication. Reusing one key for changed parameters can hide a genuinely new request. Derive or store a fingerprint and reject mismatches before dispatch.

Reconcile before retry

Recovery begins with the cheapest authoritative lookup. Query the provider by idempotency key or resource identifier. Inspect callbacks, receipts, and the external resource. Compare that evidence with the durable local journal.

The decision can be accept the observed effect, retry the same logical operation, compensate through a separately authorized effect, expire the intent, or escalate for human judgment. Each transition should record its evidence and policy basis.

Some operations provide no lookup and no safe deduplication. Make that limitation visible at admission. Require a human checkpoint, a narrower staged action, or a policy that prefers a missed operation over a duplicate one. Reliability includes knowing when automation cannot decide safely.

Recovery is replay plus reconciliation

Local state can often be rebuilt from durable events and snapshots. External state cannot be replayed into existence without consequences. Recovery therefore combines deterministic reconstruction of local facts with observation of the world.

After restart, reclaim work with a current fencing token, recheck authority and expiry, then resume from the last evidence-backed transition. Do not ask the model to infer whether an effect occurred from conversational context. The run journal and provider observations outrank prose summaries.

Test ambiguous intervals

Inject failure immediately before dispatch, after provider acceptance, before receipt persistence, during provider lookup, and after local verification. Assert that the stable key survives, parameter drift is rejected, retries respect provider scope, and the interface keeps unresolved effects open.

Also test expiry. If the provider forgets idempotency keys after a retention window, a late retry may create a second effect. Recovery policy must account for elapsed time rather than assuming the original protection remains active.

Failure boundary

Recovery fails when timeout is collapsed into success or failure, retries mint new keys, a key is reused for different parameters, provider-specific behavior is generalized into exactly-once claims, conversational memory outranks receipts, or an unresolved effect disappears during restart.

Retrieval check

A payment request timed out. The provider retains idempotency results for a bounded period, but the harness restarts after that period. It has the original key and request fingerprint but no receipt. Which observations are still useful? Under what evidence could retry be admitted, and when must the system escalate?

Sources and further reading

• Making retries safe with idempotent APIs — Amazon's first-party account of client request identifiers, semantic equivalence, and the limits of retry contracts.
• Harness Engineering Study Guide — the broader synthesis of ambiguity as a first-class runtime state.