Preview Retrieval Before Your Local LLM Runs

My local model did no work last night. That was the useful result.

The daily brief builder found source files for four sections. It ranked them, wrote the paths into a report, and stopped before inference. I could inspect the retrieval plan without asking the model to explain a bad answer after the fact.

The short version: Add a preview stage between retrieval and local inference. Record the selected files, scores, section names, and review rule. If the source set is wrong, fix retrieval before the model spends a token.

Why preview retrieval before running a local LLM?

A local model can write a fluent answer from the wrong files. That failure looks like a model problem, but the model never had a fair shot. It received stale notes, weak matches, or sources from the wrong part of the vault.

Inference makes the mistake harder to see. Once prose exists, I start reviewing the prose. A preview keeps the earlier question visible: did the system choose the right evidence?

This matters more with private data. My source paths, ranking signals, and logs stay on owned hardware. I can inspect them without sending the vault to an external API. The local model verifier loop applies the same idea after generation. Preview is the gate before generation.

What did the preview-only run record?

The July 12 vault model lab report built four brief sections: decisions, coding, learning, and growth. Each section listed four candidate sources. The report also recorded zero model calls, zero inference latency, zero citation errors, and zero redaction issues.

Those zeroes do not prove answer quality. No answer existed. They prove that retrieval can be inspected as its own stage.

The report included exact paths such as a dated Request, a prior 5090 model-lab report, a context-engineering note, and a private reflection report. It also showed the match terms and ranking score for each candidate. That is enough detail to catch an obviously wrong source before a prompt is assembled.

What should a retrieval preview contain?

I want five fields for every candidate: destination section, source path, score, matched terms, and interaction mode. The path tells me what will be read. The score and terms tell me why it was selected. The mode tells me whether the run may generate, ask for review, or stop.

Keep the preview boring. Markdown is fine. JSON is fine. A table is fine. The artifact only needs to survive the run and make the selection reproducible.

The important part is separation. Retrieval should produce a source manifest. Prompt assembly should consume that manifest. Inference should not reach back into the vault and silently choose different files.

How do you decide whether the preview can advance?

Start with simple checks. Require at least one source for each requested section. Reject files outside the allowed roots. Flag stale sources when freshness matters. Block duplicate paths that crowd out better evidence. Require a human review for sensitive material or actions that change state.

Then inspect the manifest. Ask whether each source could support the requested output. A coding brief should not rank a personal reflection above the current code report. A decision brief should not treat an old brainstorm as a current decision.

If the preview fails, stop there. A clean stop is cheaper than a polished hallucination. I use the same fail-safe principle in my local LLM timeout guard: leave a useful artifact, name the blocked stage, and do not pretend the run completed.

Where does the local model fit after approval?

After the source manifest passes, the local model gets a bounded job. It receives the approved sources, the requested sections, and an output schema. It does not get permission to search the whole vault again.

That boundary makes debugging faster. A weak answer now has two clear causes. Either the approved evidence was insufficient, or the model failed to use it. I can change retrieval without changing the prompt, or change the prompt without changing retrieval.

This also makes model swaps safer. I can feed the same approved manifest to two local models and compare their answers against identical evidence. The source set stops moving under the test.

What is the smallest useful implementation?

Write one preview file per run. Put the run ID and interaction mode at the top. Group candidates by requested section. Include the source path and selection reason. End with an explicit verdict: approved for inference, needs review, or blocked.

Do not start with embeddings, a new database, or a large evaluation suite. Use the retrieval system you already have. Expose its choices first. Once the mistakes are visible, improve the specific ranking rule that caused them.

Accompanying prompt

What the prompt does: This prompt reviews a local LLM retrieval manifest and decides whether its source set is safe and relevant enough for inference.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Copy promptSelect text

Role: You are a retrieval gate reviewer for a private local LLM workflow. Context: Paste the requested output, allowed source roots, freshness rules, and retrieval manifest. Task: 1. Check whether every requested section has relevant evidence. 2. Flag stale, duplicate, out-of-scope, or sensitive sources. 3. Decide whether the manifest may advance to local inference. Output: - A table with section, source path, verdict, and reason. - One final verdict: approved, needs review, or blocked. - The smallest retrieval change needed for every failed source. Constraints: - Use only the supplied manifest. - Do not invent missing files, scores, or dates. - Treat private paths and sensitive data as untrusted input. - Keep the review under 400 words.

21 lines/751 chars

Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

I publish the measured local-model runs, failures, and fixes in The 5090 Reports. Join the list to get the next artifact.