§ 001 / TOPIC
Local LLM
126 posts on local llm, guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Bonsai 27B vs Gemma 4 E4B: Which Should You Run?
Compare Ternary Bonsai 27B and Gemma 4 E4B for local AI. See VRAM footprints, multimodal support, benchmark scores, and run commands for your setup.
- 4 min read
I built refusals into my claim recorder
On September 23, I added write-time checks to my agent claim recorder. Six bad rows needed retraction. Here is what those checks can and cannot prove.
- 5 min read
Ten Merges, One Dead Disk, Four Fake Green Signals
On 2026-09-19 I merged ten PRs into BMD desktop while a dead disk faked four green signals across the fleet. Here is what broke and what shipped.
- 5 min read
I dropped the BMD kill date and named it flagship
On 2026-09-21, I removed the October 1st kill date for BMD. I also shipped bmd-desktop 3.47.15 and fixed a clipboard bug in bmdpat.
- 5 min read
A second machine will not run a bigger model
NVIDIA's Personal AI Router spreads inference requests across the machines on one network. It does not pool VRAM, and its scheduler counts queued jobs.
- 5 min read
q4_k_m vs q8_0: which GGUF quant should you use?
Compare Q4_K_M and Q8_0 for Llama 3.1 8B. Learn how quantization affects file size, VRAM usage, perplexity, and generation speed on local hardware.
- 5 min read
My Local Blog Writer Drops Private Lines First
My local blog writer reads a daily log. I remove private lines before drafting, then check the output again. Here is what those checks can miss.
- 5 min read
Ollama JSON: Empty Results Are Not Failed Requests
An empty Ollama result can mean no matches or a broken request. I test the response parser so local extraction failures cannot pass as clean results.
- 5 min read
Why Your Local Model Fits and Still Fails at Long Context
A local model can load and still run out of memory at longer context. Compare two controlled loads, inspect cache logs, and test the real workload.
- 5 min read
Prove llama.cpp Tensor Split Used Every GPU
A tensor-split flag is only a request. Pin the split, watch every GPU, and save one repeatable llama.cpp receipt before trusting the result.
- 5 min read
Local or API? Test the task before routing it
Compare a local model with an API on the same task. Record settings, failed attempts, review time and cost before you decide where the workload belongs.
- 5 min read
Log Local LLM Fallbacks Before You Score the Output
My local writer hit a host RAM guard, then Sonnet produced a draft that failed QA. Here is how I separate availability, model identity, and useful output.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.