Q4km vs Q5km: Q4_K_M vs Q5_K_M
q4km vs q5km is Q4_K_M vs Q5_K_M in GGUF. Pick by VRAM headroom, quality risk, and CPU fallback. Use the free quant tool before download.
Short answer: q4km and q5km are informal spellings of GGUF quants Q4_K_M and Q5_K_M. For most local LLM loads on a single consumer GPU, start with Q4_K_M. Move to Q5_K_M only when you still have VRAM after weights plus KV cache.
What do people mean by q4km vs q5km?
Search and chat often drop the underscores: q4km, q5km, q8. In llama.cpp and Ollama file names the real tags are Q4_K_M, Q5_K_M, and Q8_0 (or close cousins like Q5_K_S). Same decision: size vs quality vs speed under a fixed VRAM budget.
When should you pick Q4_K_M?
Pick Q4_K_M when:
- You want the model to load fully without CPU fallback.
- You still need context headroom for the KV cache.
- You care more about tokens/sec than the last few quality points.
Q4_K_M is the default local quant for a reason: it is usually the best balance of file size and usable quality on 8-24 GB cards.
When should you pick Q5_K_M?
Pick Q5_K_M when:
- Q4_K_M already fits with clear headroom.
- You see quality issues on coding or long-form tasks at Q4.
- You measured that Q5 still keeps full GPU offload at your context length.
If Q5 forces layers back to CPU, you often lose more quality (and speed) than you gain from the higher quant.
How do you decide in five minutes?
- Note GPU VRAM and target context length.
- Open quant compare and VRAM calculator.
- Check weight size for Q4_K_M and Q5_K_M against free VRAM after OS and cache.
- Prefer the smaller quant that stays fully on GPU.
For the full matrix (Q4/Q5/Q6/Q8, K_S vs K_M, and edge cases), read the long guide: GGUF Q4_K_M vs Q5_K_M vs Q8. For offload flags once the file fits, see llama.cpp --n-gpu-layers.
Accompanying prompt
What the prompt does: It turns your GPU VRAM, context length, and two GGUF file sizes into a Q4_K_M vs Q5_K_M pick with a hard pass/fail.
Copy/paste this prompt:
Copy-ready prompt
Paste the exact block into your coding agent.
No article chrome, no footnotes, no formatting drift.
This prompt and every other one we publish live in the free prompt library.
Copy the block above.
Weekly measured local runs: https://bmdpat.com/5090-reports
Get the local AI lab notes
Benchmark rows, VRAM fit checks, quant choices, and what actually runs on consumer GPUs. M-F, only when there is something worth sending.
Patrick Hughes
Building BMD HODL — a one-person AI-operated holding company. Nashville, Tennessee. Twenty-Two agents.
More writing
- 6 min
My 8B Model Failed a 400-Word Task
Three Llama 3.1 8B runs missed a 400-word floor. Here is the verifier-driven route that moved long-form synthesis to Gemma 4 26B.
- 5 min
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
- 5 min
How to Run Local LLM Verifier Loops on Owned Hardware
A local LLM workflow needs more than a model prompt. It needs a verifier loop that proves the file, command, URL, or report changed before the agent claims done.
- 8 min
Q4_K_M vs Q5_K_M: Which GGUF Quant?
Compare Q4_K_M vs Q5_K_M vs Q8 GGUF: VRAM fit, quality, speed, and when a smaller quant beats a bigger one that falls back to CPU.
- 7 min
llama.cpp --n-gpu-layers: -1, 0, Partial
Set --n-gpu-layers in llama.cpp with clear -1, 0, and partial offload rules, VRAM headroom checks, examples, and CPU fallback fixes.