Topic
Quantization
4 posts on quantization — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
A 7 GB 27B Model Lost to My 17 GB Default
A 6.66 GiB ternary 27B model fit my RTX 5090, but it lost the default slot. File size is only the first local model-selection gate.
- 5 min read
Q4km vs Q5km: Q4_K_M vs Q5_K_M
q4km vs q5km is Q4_K_M vs Q5_K_M in GGUF. Pick by VRAM headroom, quality risk, and CPU fallback. Use the free quant tool before download.
- 5 min read
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
- 8 min read
Q4_K_M vs Q5_K_M: Which GGUF Quant?
Compare Q4_K_M vs Q5_K_M vs Q8 GGUF: VRAM fit, quality, speed, and when a smaller quant beats a bigger one that falls back to CPU.
The AI agent build notes
Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.