[bmdpat]
Download BMD free
All writing
Updated 5 min read

Q4_K_M vs Q5_K_M (q4km vs q5km): Which to Download? (2026)

Stuck between q4km and q5km? Verdict: start with Q4_K_M on 8–24GB cards; move to Q5_K_M only if it still fits with KV-cache headroom. The 5-minute VRAM check inside.

#local-llm#gguf#quantization#llama-cpp#5090-reports
Share LinkedIn

TL;DR

  • q4km and q5km are informal spellings of GGUF quants Q4_K_M and Q5_K_M. For most local LLM loads on a single consumer GPU, start with Q4_K_M.
  • Move to Q5_K_M only when you still have VRAM after weights plus KV cache. If Q5 forces layers back to CPU, you often lose more quality and speed than you gain.
  • Q4_K_M is the default local quant for a reason: it is usually the best balance of file size and usable quality on 8-24 GB cards.
  • Prefer the smaller quant that stays fully on GPU. Check weight size for Q4_K_M and Q5_K_M against free VRAM after OS and cache.

Check my GPU

Short answer: q4km and q5km are informal spellings of GGUF quants Q4_K_M and Q5_K_M. For most local LLM loads on a single consumer GPU, start with Q4_K_M. Move to Q5_K_M only when you still have VRAM after weights plus KV cache.

What do people mean by q4km vs q5km?

Search and chat often drop the underscores: q4km, q5km, q8. In llama.cpp and Ollama file names the real tags are Q4_K_M, Q5_K_M, and Q8_0 (or close cousins like Q5_K_S). Same decision: size vs quality vs speed under a fixed VRAM budget.

When should you pick Q4_K_M?

Pick Q4_K_M when:

  1. You want the model to load fully without CPU fallback.
  2. You still need context headroom for the KV cache.
  3. You care more about tokens/sec than the last few quality points.

Q4_K_M is the default local quant for a reason: it is usually the best balance of file size and usable quality on 8-24 GB cards.

When should you pick Q5_K_M?

Pick Q5_K_M when:

  1. Q4_K_M already fits with clear headroom.
  2. You see quality issues on coding or long-form tasks at Q4.
  3. You measured that Q5 still keeps full GPU offload at your context length.

If Q5 forces layers back to CPU, you often lose more quality (and speed) than you gain from the higher quant.

How do you decide in five minutes?

  1. Note GPU VRAM and target context length.
  2. Open quant compare and VRAM calculator.
  3. Check weight size for Q4_K_M and Q5_K_M against free VRAM after OS and cache.
  4. Prefer the smaller quant that stays fully on GPU.

For the full matrix (Q4/Q5/Q6/Q8, K_S vs K_M, and edge cases), read the long guide: GGUF Q4_K_M vs Q5_K_M vs Q8. For offload flags once the file fits, see llama.cpp --n-gpu-layers.

Accompanying prompt

What the prompt does: It turns your GPU VRAM, context length, and two GGUF file sizes into a Q4_K_M vs Q5_K_M pick with a hard pass/fail.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Role: You are a local LLM quant picker. Context: GPU VRAM (GB): [ ] Target context length: [ ] Model: [ ] Q4_K_M file size (GB): [ ] Q5_K_M file size (GB): [ ] Need full GPU offload: yes/no Task: 1. Estimate free VRAM after OS and KV cache. 2. Say whether Q4_K_M and Q5_K_M fit fully on GPU. 3. Pick one quant and one fallback if load fails. 4. List the single next measurement to run. Output: - Pick: Q4_K_M or Q5_K_M - Fit: full GPU / mixed / no - Fallback plan - Next command or check Constraints: - Prefer full GPU offload over a higher quant with CPU fallback. - Do not invent sizes I did not provide. - Keep the answer under 12 lines.
27 lines644 chars
Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

Weekly measured local runs: https://bmdpat.com/5090-reports

FAQ

What do people mean by q4km vs q5km?

Search and chat often drop the underscores. In llama.cpp and Ollama file names the real tags are Q4_K_M, Q5_K_M, and Q8_0 or close cousins like Q5_K_S.

When should you pick Q4_K_M?

When you want the model to load fully without CPU fallback, you still need context headroom for the KV cache, and you care more about tokens/sec than the last few quality points.

When should you pick Q5_K_M?

When Q4_K_M already fits with clear headroom, you see quality issues on coding or long-form tasks at Q4, and you measured that Q5 still keeps full GPU offload at your context length.

Get the Local AI Field Kit

Four copy-ready tools now, then one evidence-backed Local AI Lab Note on Friday when there is something worth sharing.

Try the free agent run check first

Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.

PH

Patrick Hughes

I build BMD and publish measured AI runs, failure reports, and reusable checks. Nashville, Tennessee.

More writing