Topic
LLM Benchmarks
5 posts on llm benchmarks — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
When a 4B Local LLM Beats 26B on One Task
On one RTX 5090 workshop, a 4B model beat a 26B model on speed while both passed four code checks. Here is the model-selection rule I kept.
- 5 min read
The faster local model run took 83x longer
One llama3.1:8b run posted 188.16 tok/s and finished in 43.9 seconds. The next posted 153.12 and finished in 0.53. Tokens per second ranked them backwards.
- 5 min read
My 5090 benchmark was missing the field I needed most
A fresh Qwen3.5 9B run showed 84.94 tok/s, but the useful number was the 6,105 ms load phase. I added phase timings and capture time to the benchmark receipt.
- 6 min read
How I Keep LLM Results Valid After a Driver Update
A new GPU snapshot does not refresh an old benchmark. I show how I bind driver, runtime, workload, and timestamps to each local LLM result.
- 6 min read
Your local LLM benchmark is probably lying to you
A local model pass rate can be true and useless at the same time. Here are the three ways local LLM benchmarks mislead you, drawn from real rows on my RTX 5090.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.