Topic
Benchmarking
7 posts on benchmarking — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Long Context Cost Me VRAM, Not Tokens a Second
A 94x bigger prompt cost llama3.1:8b only 9.7% of its generation rate. VRAM rose 656 MiB, but num_ctx doubled in the same step. Budget memory first.
- 5 min read
One model logged 14.5 and 6,178 prompt tokens a second
gemma4:26b read 53 prompt tokens at 14.5 tok/s and 4,470 at 6,178.9 on the same RTX 5090 run set. Prompt speed is not one number.
- 5 min read
Your Local LLM CSV Needs a Schema Version
I audited seven RTX 5090 benchmark CSVs and found four header widths, three power schemas, and no schema version. Here is the row contract I use now.
- 5 min read
Your Benchmark Row Never Saved the Driver Version
I audited seven RTX 5090 benchmark files on 2026-08-27. Not one records the GPU driver version. Here are the four fields every local LLM row needs.
- 5 min read
The 26B Model Hit the Cap. The 8B Finished.
Same code task, same 512-token cap. Ollama's done_reason showed one local model finished and one got cut off. Tokens per second hid it.
- 5 min read
How I Benchmark Local LLMs Before I Trust Them
A local LLM benchmark should end with a decision. I record task quality, tokens per second, VRAM, power, and failure state before I add a model.
- 6 min read
Why I Benchmark Local LLM Input and Output Separately
My RTX 5090 runs show why model loading, prompt ingestion, token generation, and task checks need separate measurements before production use.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.