Topic
Hardware
40 posts on hardware — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
My local LLM eval hid four token caps
My 5090-rig eval uses four output caps: 50, 180, 192, and 256 tokens. A score without the cap is not a model result. Record the cap on every row.
- 5 min read
A GPU Driver Is Not a Local LLM Benchmark
My 2026-08-20 RTX 5090 report lists driver 610.88 and 17 benchmark rows. That does not show the driver changed model speed.
- 5 min read
The 26B Model Hit the Cap. The 8B Finished.
Same code task, same 512-token cap. Ollama's done_reason showed one local model finished and one got cut off. Tokens per second hid it.
- 5 min read
A 27B Model Fit on an 8 GB GPU. It Was Slow.
Qwen3.8-27B Q3_K_S loaded on an RTX 3070. VRAM used 7,435 of 8,192 MiB. Decode ran 2.07 tok/s. Fit on 8 GB is not a usable rate.
- 4 min read
My Local LLM Writer Failed Its Own Word-Count Gate
Gemma 4 26B wrote a 535-word draft against a 550-word floor. The gate blocked it, both rescue engines were down, and no bad post shipped.
- 5 min read
How I Benchmark Local LLMs Before I Trust Them
A local LLM benchmark should end with a decision. I record task quality, tokens per second, VRAM, power, and failure state before I add a model.
- 5 min read
How I Test a 30B Local Model Before I Load It
A 30B local model can fit on paper and still fail the job. This test plan checks memory, tool use, speed, and repeatability first.
- 5 min read
When a 4B Local LLM Beats 26B on One Task
On one RTX 5090 workshop, a 4B model beat a 26B model on speed while both passed four code checks. Here is the model-selection rule I kept.
- 5 min read
My Local LLM Got Faster After It Passed the Tests
A three-run RTX 5090 test showed why local LLM tuning must pair speed with fixed-task checks. One faster setting also repaired every test.
- 6 min read
My local models refused zero of 50 security tasks
Hosted coding assistants have declined defensive security work. I ran 50 such tasks across five local models on my own hardware and counted zero refusals.
- 5 min read
The faster local model run took 83x longer
One llama3.1:8b run posted 188.16 tok/s and finished in 43.9 seconds. The next posted 153.12 and finished in 0.53. Tokens per second ranked them backwards.
- 5 min read
My 5090 benchmark was missing the field I needed most
A fresh Qwen3.5 9B run showed 84.94 tok/s, but the useful number was the 6,105 ms load phase. I added phase timings and capture time to the benchmark receipt.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.