Search Old Results Before Publishing an LLM Test

My latest RTX 5090 post passed every mechanical check and still failed QA. I had already published the finding.

The short answer: Before I write from a local LLM benchmark, I now search the published archive for the model, runtime setting, and two signature numbers from the result. That check catches a repeated conclusion even when the title and framing changed. On August 1, 2026, it stopped me from publishing a second post about the same Ollama num_batch sweep. Canonical URL: https://bmdpat.com/blog/ollama-num-batch-tuning-quality-gate-2026

How did the duplicate get through the first checks?

The new draft looked clean. It used a fresh July 30 report, cited the source rows, included two live internal links, passed the prose gate, passed the publish-body validator, and carried a generator-made visual that cleared the mechanical image check. Every one of those gates reads the draft on its own. None of them reads the archive.

Its conclusion was also defensible. In the saved gemma4:26b workshop rows, num_batch=256 produced 215.7 output tokens per second at 187.6 watts average. The 512 row reached 218.5 output tokens per second at 234.3 watts average. Both passed the fixed task checks.

The problem was not truth. The problem was novelty.

My July 19 post, Ollama num_batch: 256 Was My RTX 5090 Sweet Spot, had already compared those rows and made the same route choice. The second draft changed the title and added a tuning checklist, but the load-bearing finding was identical.

Why is a new report date not a new result?

The July 30 lab report carries generated_at: 2026-07-30 09:50. That stamp records when the rollup ran. It says nothing about when any single row was measured.

The saved rows prove the gap. The workshop benchmark CSV has these columns: model, quant, engine, workload, input tokens, output tokens, tokens per second, watts average, and notes. There is no capture-time column at all. The same July 30 table also mixes runtime versions, listing rows from Ollama 0.22.1 and rows from Ollama 0.31.1. One report date cannot stand in for the run time of any row.

So a report generator can pull an old measurement back into view without creating new evidence. If I treat the newest report as a list of new findings, I publish the same result twice.

My benchmark provenance guide argues for binding driver, runtime, model, workload, settings, and capture time to every row. This workshop CSV predates that rule, which is exactly how a stale row reads as current. The content path needs one more field on top: the public claim already made from that row.

What is a signature-number search?

A title search is too weak. The old post may use different words.

I search for three parts of the result together:

1. The model or model family, such as gemma4:26b.
2. The changed setting, such as num_batch.
3. Two numbers that identify the comparison, such as 215.7 and 218.5.

The numbers act like a small fingerprint. They survive headline changes and topic labels. A match does not always mean the new post is a duplicate, but it forces a direct comparison before drafting continues.

For this run, the search found the July 19 post. Independent QA then compared the thesis, rows, power tradeoff, and recommendation. It returned FAIL, so the post did not ship.

When can the same benchmark support another post?

The bar is a new decision, not a new introduction.

The same rows can support a follow-up if new evidence changes the route, a later runtime version changes the result, or the post answers a different operational question that the first one did not answer. A translation of the same conclusion into another checklist is not enough.

For example, a repeated sweep across several seeds could show whether the 2.8 tokens-per-second gap persists. A new VRAM trace could test whether either batch value crosses a memory limit. A queue-level run could measure completed jobs per hour instead of output rate. Those would add evidence.

Without that addition, I should link the old post and move on.

What gate do I run before drafting now?

First, identify the exact measurement rows. Do not start from the report title.

Second, extract the model, setting, workload, and two signature numbers. Search the published archive for all four.

Third, compare the proposed one-sentence takeaway with each match. If the new post reaches the same conclusion from the same rows, stop.

Fourth, record what changed. A new model, rerun, verifier, route decision, or failure can justify a follow-up. A new date in a rollup cannot.

This gate runs before I draft the hook. It also keeps the public lab notebook honest. Readers get one clear result instead of several versions that make the evidence look larger than it is.

Accompanying prompt

What the prompt does: It checks whether a planned local LLM benchmark post repeats a finding that is already public.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Copy promptSelect text

Role: You are checking a local LLM post for duplicate evidence. Context: Paste the planned takeaway, source benchmark rows, and a list of published post titles, URLs, and text. Task: 1. Extract the model, setting, workload, and two signature numbers. 2. Search the published text for matching evidence. 3. Compare the old and planned one-sentence takeaways. 4. State what new measurement or decision the planned post adds. Output: - PASS if the post adds a distinct finding. - FAIL if it repeats the same conclusion from the same rows. - The closest prior post and the exact overlap. Constraints: - Do not treat a report generation date as a measurement date. - Do not approve a rewrite that only changes the title or checklist. - Keep measured facts separate from proposed follow-up tests. - Do not invent source rows, numbers, or run dates.

23 lines/846 chars

Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

I publish measured local AI build notes in The 5090 Reports. Join the email list for the next result.