Skip to content
[bmdpat]
All writing
5 min read

The e2e run passed on 2026-11-07 after eleven red nights

On 2026-11-07, an eleven-night test streak cleared, nine lines unblocked the newsletter, and an agent cloned its own body under a new title.

ShareLinkedIn

No bmdpat workflow failed in 36 hours before the morning of 2026-11-07. That ended an eleven-night run of broken test assertions. On 2026-11-06 I left two falsifiable predictions in place. On 2026-11-07 I graded both. One prediction cleared. The other broke in half.

Key decisions from The e2e run passed on 2026-11-07 after eleven red nightsOpen full-size image (opens in new tab)

Why did the eleven-night test streak break?

Your test suite can pass while your background jobs sit broken. On 2026-11-07, the continuous e2e assertion passed because no workflow crashed in 36 hours. You can clear your core integration test, but you still need to verify which background workers actually ran.

You know the feeling when red builds stack up. You open your console and expect red text. On 2026-11-07 the build went green. Yet green did not mean your system worked. Doctor stayed red overnight on 2026-11-07. It tracked 81 expected jobs. Only 13 fired. 54 jobs sat paused, and 1 went missing.

Your spend guard can fail quietly too. On 2026-11-07, the budget gate reported fail_open_no_data. When your telemetry drops, your kill switch cannot fire. That keeps work running, but you lose AI agent cost control. If you manage autonomous callers, you need a ceiling that halts your runs when data drops.

You can compare how the five operational lanes behaved on 2026-11-07:

System LaneState on 2026-11-07Blocked By
e2e workflowsGreen (0 failures in 36h)None
Newsletter queueGreen at commit 5fb4c26bcbStale packet lock
SecurityAnalystRed (5 P0, 29 P1, 41 P2)2 keys in git history
Doctor monitorRed (13 fired, 54 paused)Paused PR queue builder
Backup mirrorRed (rc=9, 28 dirs failed)Unmounted backup disk

How did nine lines unblock weeks of drafts?

A queue can freeze if your code checks memory flags instead of actual files on your disk. On 2026-11-07, nine net lines of code and 26 test lines unblocked weeks of queued drafts. Your code must reconcile disk state before you let a checker halt your queue.

You can inspect the logic that froze the pipeline:

def find_blocking_packet(packet_path: str, packet_status: str) -> bool: disk_record = read_packet_record(packet_path) if disk_record.is_rejected and packet_status != "closed": reconcile_packet_state(packet_path, target="closed") return False return packet_status == "blocking"

The newsletter queue stalled on the packet for 2026-W38. Every later weekly draft stood behind it. The function find_blocking_packet read packet_status without reconciling the file. A rejection written after the fact never closed the row. At 00:27 CT on 2026-11-07, commit 5fb4c26bcb shipped. Nine lines of code flushed your backlog.

One post went live right after that fix. "A stop order closed REAL for 21.56% on 2026-11-05" published at sha256 6e3c427c1cca. Gemini 3.8 Flash drafted it. The automated QA pass needed one repair. Four publish attempts failed before the server returned HTTP 200. You can benchmark local model latency with my 5090 local inference guide. When your network drops calls, local fallbacks protect your stack.

Why do agents report failure after finishing?

An agent will report a failure code when secondary path commands fail, even after your main files sync. On 2026-11-07, pathspec errors on archived files triggered red alarms while the git remote sat clean. You must check primary command outputs before you trust an agent error code.

On 2026-11-07, the log digest showed vault-push failing 21 times. On 2026-11-06 it logged 24 failures. The agent tried to match a pathspec for an archived document. Git reported an error on the missing pattern, yet the same row showed push exit=0. Origin held the commits. Git had the code. The backup mirror failed because robocopy returned rc=9 across 28 directories.

SecurityAnalyst also half-passed on 2026-11-07. It produced its first self-describing title in 17 runs. Then it generated a body with hash 22c802e66620, matching the text from 2026-11-06 byte for byte. It fixed the heading and cloned the payload. The ticket queue still carries two private keys in repository history on day nineteen. You cannot trust summary status text until you run BMD, see what your agents actually did. Your agent can announce a failure while saving your work, or report a pass while printing duplicate noise.

The log digest found vault-push reporting fail 24 times a day on a pathspec for an archived file, while the same row says push exit=0. On 2026-11-07 it failed 21 more times and the remote is current. Doctor counts those rows and so do I. A real share of the RED I read each morning is an agent reporting it failed at something it finished.

What should you do with this?

  1. Inspect the exit code logic in your runner to verify trailing pathspec warnings do not mask a clean push in your repo.
  2. Add a disk reconciliation step to your queue monitors so an unread file rejection never blocks your new tasks behind stale records.
  3. Check the content hash of your generated agent files to catch runs that produce duplicate text under new titles.

Accompanying prompt

What the prompt does: Audits agent execution logs to separate clean completions from false failure alerts caused by missing paths.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Role: Log Auditor Context: An automation agent reported an error status during a synchronization task, but the remote destination might contain the finished commit. Inputs: - Log text: __ - Remote branch status: __ - Expected artifact name: __ Task: 1. Parse the log text for successful command steps. 2. Check if the remote branch status confirms receipt of the expected artifact name. 3. Identify whether errors relate to secondary pathspec lookups or primary push commands. 4. Output the true state of the run. Output: - Final job status (CLEAN, FALSE_ALARM, or BROKEN) - Culprit log line - Verification action Constraints: - Rely only on supplied inputs. - Do not assume missing files caused data loss if exit code is 0.
25 lines729 chars
Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

Weekly measured local runs: https://bmdpat.com/5090-reports

Get the Local AI Field Kit

Four copy-ready tools now, then one evidence-backed Local AI Lab Note on Friday when there is something worth sharing.

Try the free agent run check first

Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.

Written by

Patrick Hughes

I build BMD and publish measured AI runs, failure reports, and reusable checks. Nashville, Tennessee.

More writing