Topic
Quantization
4 posts on quantization — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
A 7 GB 27B Model Lost to My 17 GB Default
A 6.66 GiB ternary 27B model fit my RTX 5090, but it lost the default slot. File size is only the first local model-selection gate.
- 5 min read
Q4km vs Q5km: Q4_K_M vs Q5_K_M
q4km vs q5km is Q4_K_M vs Q5_K_M in GGUF. Pick by VRAM headroom, quality risk, and CPU fallback. Use the free quant tool before download.
- 5 min read
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
- 8 min read
GGUF Quantization Explained: Q4 vs Q5 vs Q8 (2026)
Downloading a GGUF and unsure which quant? Q4_K_M, Q5_K_M, and Q8 trade quality for VRAM — and a smaller quant can beat a bigger one that spills to CPU. The full ladder, explained. (2026)
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.