Topic
Hardware
57 posts on hardware, guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Bonsai 27B vs Gemma 4 E4B: Which Should You Run?
Compare Ternary Bonsai 27B and Gemma 4 E4B for local AI. See VRAM footprints, multimodal support, benchmark scores, and run commands for your setup.
- 5 min read
A second machine will not run a bigger model
NVIDIA's Personal AI Router spreads inference requests across the machines on one network. It does not pool VRAM, and its scheduler counts queued jobs.
- 5 min read
q4_k_m vs q8_0: which GGUF quant should you use?
Compare Q4_K_M and Q8_0 for Llama 3.1 8B. Learn how quantization affects file size, VRAM usage, perplexity, and generation speed on local hardware.
- 5 min read
Why Your Local Model Fits and Still Fails at Long Context
A local model can load and still run out of memory at longer context. Compare two controlled loads, inspect cache logs, and test the real workload.
- 5 min read
Local or API? Test the task before routing it
Compare a local model with an API on the same task. Record settings, failed attempts, review time and cost before you decide where the workload belongs.
- 5 min read
Log Local LLM Fallbacks Before You Score the Output
My local writer hit a host RAM guard, then Sonnet produced a draft that failed QA. Here is how I separate availability, model identity, and useful output.
- 5 min read
My local daily brief used 0 model calls
On September 4, 2026 my llama3.1:8b daily brief was ready with 0 model calls and 0 seconds. Preview-only file ranking is not a model run.
- 5 min read
Long Context Cost Me VRAM, Not Tokens a Second
A 94x bigger prompt cost llama3.1:8b only 9.7% of its generation rate. VRAM rose 656 MiB, but num_ctx doubled in the same step. Budget memory first.
- 5 min read
One model logged 14.5 and 6,178 prompt tokens a second
gemma4:26b read 53 prompt tokens at 14.5 tok/s and 4,470 at 6,178.9 on the same RTX 5090 run set. Prompt speed is not one number.
- 5 min read
Ollama Load Time Can Hide a Fast Local LLM
My RTX 5090 run spent 43.48 seconds on model load and 0.17 seconds on output. Split wall time before you judge local LLM speed.
- 5 min read
A 32 GB GPU Still Needs Host RAM Headroom
My RTX 5090 had 32 GB of VRAM ready, but my local policy stopped Ollama at 7.02 GiB of free host RAM. Here is my preflight before model load.
- 5 min read
Your Local LLM CSV Needs a Schema Version
I audited seven RTX 5090 benchmark CSVs and found four header widths, three power schemas, and no schema version. Here is the row contract I use now.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.