Topic
Hardware
54 posts on hardware — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
The faster local model run took 83x longer
One llama3.1:8b run posted 188.16 tok/s and finished in 43.9 seconds. The next posted 153.12 and finished in 0.53. Tokens per second ranked them backwards.
- 5 min read
My 5090 benchmark was missing the field I needed most
A fresh Qwen3.5 9B run showed 84.94 tok/s, but the useful number was the 6,105 ms load phase. I added phase timings and capture time to the benchmark receipt.
- 6 min read
Search Old Results Before Publishing an LLM Test
An independent QA pass caught my second post about the same Ollama batch sweep. Here is the duplicate check I now run before publishing an LLM result.
- 6 min read
How I Keep LLM Results Valid After a Driver Update
A new GPU snapshot does not refresh an old benchmark. I show how I bind driver, runtime, workload, and timestamps to each local LLM result.
- 6 min read
Why I Benchmark Local LLM Input and Output Separately
My RTX 5090 runs show why model loading, prompt ingestion, token generation, and task checks need separate measurements before production use.
- 6 min read
How I Budget VRAM for Shared Local AI Workloads
My RTX 5090 measurements show how I budget VRAM for a local model, a real workload, and the GPU processes that must stay resident beside it.
- 5 min read
Why I Test 3 Workloads Before Sizing a Local LLM
One local LLM speed number hides the work behind it. My RTX 5090 sweep shows why short generation, long context, and code need separate rates.
- 6 min read
Your local LLM benchmark is probably lying to you
A local model pass rate can be true and useless at the same time. Here are the three ways local LLM benchmarks mislead you, drawn from real rows on my RTX 5090.
- 5 min read
Test Retrieval Before Your Local LLM Writes
I make my local LLM show its source plan before it writes. The preview gate catches weak retrieval while the fix is still cheap and easy to inspect.
- 5 min read
Why I Did Not Promote My Smaller Local Model
My smaller local model existed and ran, but it did not beat the baseline. Here is the promotion gate I use before changing a working local AI route.
- 6 min read
Build an AI Research Workbench on Your Own GPU
Claude Science keeps data on local systems but sends selected context to Claude. Here is the owned-GPU version, with measured RTX 5090 costs.
- 5 min read
Ollama num_batch: 256 Was My RTX 5090 Sweet Spot
I tested four Ollama settings on Gemma 4 26B. num_batch 256 delivered nearly all the speed of 512 while using less power on my RTX 5090.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.