Your Agent's Audit Trail Cannot Be Retrofitted

My agent said it completed three actions. Two had not happened.

The chat transcript looked fine. The tool trace showed activity. The final answer sounded certain. None of that proved the files and state matched the answer.

That failure led me to build a separate verification layer. Every material completion claim now needs a deterministic check. The session cannot close cleanly when a RED claim fails.

Logs are not outcome proof

An agent trace can show that a model called a file-writing tool. It cannot prove the write landed in the right repository, contained the required value, passed the relevant test, or survived another process changing the file.

Those are outcome questions.

I wanted each claim to be falsifiable. "Updated the configuration" became a file-content check. "Moved the queue task" became a source-absent and destination-present check. "Tests pass" became a locked command with an expected exit code.

No model grades the model. The checker reads reality.

The record has three parts

The first part is the claim:

{
 "session": "deploy-fix",
 "claim": "the API timeout is configured",
 "severity": "RED",
 "artifact": "config/api.yaml",
 "check": {
 "type": "file_contains",
 "path": "config/api.yaml",
 "pattern": "timeout: 30"
 }
}

The second part is deterministic verification. The current checker set covers file existence, file content, path moves, frontmatter, glob counts, and locked Python commands. Vacuous checks fail. A regex that matches every file is not proof. A count of at least zero is not proof.

The third part is an append-only receipt. Corrections become retraction records. The original assertion remains in history.

The exit gate changes agent behavior

A normal stop hook runs after the agent has already stopped. It can record a verdict, but it cannot send the agent back to fix its work.

The explicit finish command can.

showwork finish --session deploy-fix --status ok

If a RED claim fails, the command exits with code 2 and refuses the clean close. The agent must fix the outcome, retract the claim truthfully, or report the session as blocked.

The Stop-hook adapter still records the final verdict. It always exits zero because it observes rather than gates. Keeping those roles separate prevents a reporting hook from becoming an availability risk.

The production ledger is not perfectly green

The ledger runs next to the agents themselves: my fleet operates on owned hardware, and the receipts live on the same machine as the work. The sanitized source ledger contains 2,158 claims from 842 sessions. Deterministic checks back 2,152 claims. It also contains 152 retraction records and one malformed line that the parser surfaced instead of dropping.

The captured audit was RED at 54 of 60 verified.

That is useful evidence. A verification system that converts its own gaps into a green launch number is not doing verification.

The public case study contains aggregates only. It excludes claim text, private paths, strategy, financial values, and trading records. A SHA-256 fingerprint ties the aggregate to the exact source ledgers used for the derivation.

The AI Act makes evidence an engineering problem

The European Commission says the AI Act becomes fully applicable on 2 August 2026, with named exceptions and longer transitions for some regulated-product systems. That does not make this package a compliance solution. I am not offering legal advice.

It does make one engineering question harder to ignore: can you reproduce the evidence behind an agent's claimed outcome?

Action logs are useful. Model traces are useful. Neither replaces a check that compares the assertion with the resulting state.

The official application timeline is on the European Commission's AI Act page.

The portable part is the specification

The Python package is one implementation. The longer-lived artifact is spec-v0.1, which defines the JSONL record, six check types, retractions, verdict rules, session events, and exit-gate behavior.

Someone can implement the same format in Go or TypeScript without reading the Python source.

The repository is bmdhodl/showwork. It includes the Stop-hook adapter, conformance tests, genesis receipt, and sanitized production case study.

Bound the agent before you audit it

A receipt proves what happened. It does not cap what an agent can spend on the way to a wrong outcome. Evidence and limits are two sides of one problem: trusting work no human watched in real time.

AgentGuard is the limit side. It puts hard ceilings on tokens, cost, and call rate around an agent run and enforces them while the run is live, so a looping agent hits a wall instead of your bill. showwork is the evidence side: the append-only receipt that proves the outcome after the fact. Bound what the agent can do, then prove what it did.

Related reading

• My Agents Have to Prove What They Did
• Pin Your Context Window or Pay the Reload Tax

Accompanying prompt

What the prompt does: audits an existing agent workflow for outcome claims that today rest only on logs, and designs the receipt layer to close each gap.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Copy promptSelect text

Role: platform engineer hardening an AI agent workflow. Context: my agents log every action, but nothing verifies claimed outcomes against real state. The workflow description follows this prompt. Task: 1. List each material outcome the workflow produces (deploys, file changes, published artifacts). 2. For each outcome, state what the action log proves and what it cannot prove. 3. Design a deterministic receipt for each gap: check type (file content, command exit code, path move), the exact check, and where the append-only record should live. 4. Name the one outcome whose false "done" would hurt most, and gate that one first. Output: a gap table plus an ordered rollout plan for the receipt layer. Constraints: checks must be deterministic and runnable today. No model-graded evidence.

9 lines/793 chars

Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

Get the artifact-backed local AI lab notes by email: https://bmdpat.com/5090-reports