Q4km vs Q5km: Q4_K_M vs Q5_K_M

Short answer: q4km and q5km are informal spellings of GGUF quants Q4_K_M and Q5_K_M. For most local LLM loads on a single consumer GPU, start with Q4_K_M. Move to Q5_K_M only when you still have VRAM after weights plus KV cache.

What do people mean by q4km vs q5km?

Search and chat often drop the underscores: q4km, q5km, q8. In llama.cpp and Ollama file names the real tags are Q4_K_M, Q5_K_M, and Q8_0 (or close cousins like Q5_K_S). Same decision: size vs quality vs speed under a fixed VRAM budget.

When should you pick Q4_K_M?

Pick Q4_K_M when:

1. You want the model to load fully without CPU fallback.
2. You still need context headroom for the KV cache.
3. You care more about tokens/sec than the last few quality points.

Q4_K_M is the default local quant for a reason: it is usually the best balance of file size and usable quality on 8-24 GB cards.

When should you pick Q5_K_M?

Pick Q5_K_M when:

1. Q4_K_M already fits with clear headroom.
2. You see quality issues on coding or long-form tasks at Q4.
3. You measured that Q5 still keeps full GPU offload at your context length.

If Q5 forces layers back to CPU, you often lose more quality (and speed) than you gain from the higher quant.

How do you decide in five minutes?

1. Note GPU VRAM and target context length.
2. Open quant compare and VRAM calculator.
3. Check weight size for Q4_K_M and Q5_K_M against free VRAM after OS and cache.
4. Prefer the smaller quant that stays fully on GPU.

For the full matrix (Q4/Q5/Q6/Q8, K_S vs K_M, and edge cases), read the long guide: GGUF Q4_K_M vs Q5_K_M vs Q8. For offload flags once the file fits, see llama.cpp --n-gpu-layers.

Accompanying prompt

What the prompt does: It turns your GPU VRAM, context length, and two GGUF file sizes into a Q4_K_M vs Q5_K_M pick with a hard pass/fail.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Copy promptSelect text

Role: You are a local LLM quant picker. Context: GPU VRAM (GB): [ ] Target context length: [ ] Model: [ ] Q4_K_M file size (GB): [ ] Q5_K_M file size (GB): [ ] Need full GPU offload: yes/no Task: 1. Estimate free VRAM after OS and KV cache. 2. Say whether Q4_K_M and Q5_K_M fit fully on GPU. 3. Pick one quant and one fallback if load fails. 4. List the single next measurement to run. Output: - Pick: Q4_K_M or Q5_K_M - Fit: full GPU / mixed / no - Fallback plan - Next command or check Constraints: - Prefer full GPU offload over a higher quant with CPU fallback. - Do not invent sizes I did not provide. - Keep the answer under 12 lines.

27 lines/644 chars

Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

Weekly measured local runs: https://bmdpat.com/5090-reports