Topic
Quantization
5 posts on quantization — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
- 5 min read
GGUF Quant Cheat Sheet: Q4 vs Q5 vs Q6 vs Q8 (2026)
Skip the theory — a one-glance decision table for Q4_K_M, Q5_K_M, Q6_K, and Q8_0 on consumer GPUs, with the quality and size tradeoff spelled out for each.
- 5 min read
GGUF Quantization and VRAM: How to Pick Q4, Q5, or Q8 for Your GPU (2026)
VRAM decides your GGUF quant, not vibes. How I assign Q4, Q5, Q8 across an 8GB 3070, 16GB 5070 Ti, and 32GB 5090.
- 6 min read
How to Pick a GGUF Quant Level for Your VRAM Budget
Given your GPU, which GGUF quant do you actually pick? The VRAM math, a card-by-card table, and the quality tradeoff in plain terms.
- 8 min read
GGUF Quantization 2026: Q4_K_M vs Q5 vs Q8 — Which to Pick
Short answer: Q4_K_M wins for most local LLMs — 75% smaller with near-zero quality loss. Q5, Q6 and Q8 each win edge cases. Benchmarked on real GPUs — here's the pick for your VRAM. (2026)
The AI agent build notes
Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.