Topic
Local LLM
90 posts on local llm, guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Agent Memory: Test the Answer After a Correction
Our agent-memory replay checks words in a file. I show what that proves, what it misses, and how to test whether an expert correction changes an answer.
- 5 min read
My local daily brief used 0 model calls
On September 4, 2026 my llama3.1:8b daily brief was ready with 0 model calls and 0 seconds. Preview-only file ranking is not a model run.
- 5 min read
Long Context Cost Me VRAM, Not Tokens a Second
A 94x bigger prompt cost llama3.1:8b only 9.7% of its generation rate. VRAM rose 656 MiB, but num_ctx doubled in the same step. Budget memory first.
- 5 min read
3 Tests Before a GGUF Quant Runs Your Coding Agent
A GGUF file fitting in VRAM does not prove it can run your coding agent. Use this local acceptance test for tools, patches, and repeat runs.
- 5 min read
One model logged 14.5 and 6,178 prompt tokens a second
gemma4:26b read 53 prompt tokens at 14.5 tok/s and 4,470 at 6,178.9 on the same RTX 5090 run set. Prompt speed is not one number.
- 5 min read
Split Local LLM Prose and JSON Jobs by Model
My local AI stack routes prose to Gemma and bounded JSON to Qwen. The split keeps model choice tied to the output contract, not one leaderboard.
- 5 min read
Ollama Load Time Can Hide a Fast Local LLM
My RTX 5090 run spent 43.48 seconds on model load and 0.17 seconds on output. Split wall time before you judge local LLM speed.
- 5 min read
A 32 GB GPU Still Needs Host RAM Headroom
My RTX 5090 had 32 GB of VRAM ready, but my local policy stopped Ollama at 7.02 GiB of free host RAM. Here is my preflight before model load.
- 5 min read
Your Local LLM CSV Needs a Schema Version
I audited seven RTX 5090 benchmark CSVs and found four header widths, three power schemas, and no schema version. Here is the row contract I use now.
- 5 min read
Your Benchmark Row Never Saved the Driver Version
I audited seven RTX 5090 benchmark files on 2026-08-27. Not one records the GPU driver version. Here are the four fields every local LLM row needs.
- 5 min read
Preflight Local AI Before You Benchmark a Model
My local AI test stopped before model load because the runtime could not build. This preflight keeps setup failures out of benchmark results.
- 5 min read
49W Average Hid a 338W Burst on Gemma 26B
Gemma 4 26B Q4_K_M averaged 49 W on a long RTX 5090 run and peaked at 338 W. Keep both watt numbers before you compute energy per token.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.