Topic
Local LLM
54 posts on local llm — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 8 min read
GGUF Quantization Explained: Q4 vs Q5 vs Q8 (2026)
Downloading a GGUF and unsure which quant? Q4_K_M, Q5_K_M, and Q8 trade quality for VRAM — and a smaller quant can beat a bigger one that spills to CPU. The full ladder, explained. (2026)
- 5 min read
GPU Prices Up 48% in Two Months. I Run LLMs in My Garage.
Blackwell rental hit $4.08/hr. CoreWeave raised prices 20%. Anthropic restricted their newest model to 40 orgs. Meanwhile, consumer GPUs are sitting idle.
- 6 min read
Anthropic's Advisor Tool Is the Cost-Split Pattern You Should Already Be Running
Anthropic shipped a pattern where a cheap model runs the loop and escalates to Opus only when it needs to. The pattern works on any two-model setup. Here is the math and the playbook.
- 7 min read
llama.cpp --n-gpu-layers: -1, 0, Partial
Set --n-gpu-layers in llama.cpp with clear -1, 0, and partial offload rules, VRAM headroom checks, examples, and CPU fallback fixes.
- 6 min read
Raspberry Pi 5 Offline Voice Assistant: Sub-2s, No Cloud (2026)
Want a private voice assistant with zero cloud and no subscription? A Raspberry Pi 5 runs it offline at sub-2s latency. We tested 6 local models on real hardware — here's the winner. (2026)
- 7 min read
Local LLM on Consumer GPUs: 50 req/s, $0/Call [Benchmarks 2026]
Cloud LLM bills hit $2K/month fast. An RTX 5070 Ti serves Llama 3.1 at 50 req/s for $0 per call — we benchmarked 4 consumer GPUs and built the exact production setup.
The AI agent build notes
Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.