Topic
Llama CPP
9 posts on llama cpp — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Does Ollama Include That New llama.cpp Feature?
A llama.cpp release note does not prove that Ollama can use the feature. I trace the active runtime path, pin versions, and test the same workload.
- 5 min read
Q4km vs Q5km: Q4_K_M vs Q5_K_M
q4km vs q5km is Q4_K_M vs Q5_K_M in GGUF. Pick by VRAM headroom, quality risk, and CPU fallback. Use the free quant tool before download.
- 5 min read
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
- 5 min read
How to Run Local LLM Verifier Loops on Owned Hardware
A local LLM workflow needs more than a model prompt. It needs a verifier loop that proves the file, command, URL, or report changed before the agent claims done.
- 7 min read
llama.cpp --tensor-split Documentation: 70B Across 2 GPUs
The multi-GPU reference for llama.cpp: --tensor-split ratios, --split-mode layer vs row, --main-gpu, VRAM math, and a real two-card rig running a 70B.
- 6 min read
llama.cpp -ngl 99 Still on CPU? 5 Fixes, Ranked (2026)
You set -ngl 99 and llama.cpp still pins the CPU — the flag isn't the bug. Here's the 30-second load-log check and the 5 real causes, ranked by how often they bite.
- 8 min read
GGUF Quantization Explained: Q4 vs Q5 vs Q8 (2026)
Downloading a GGUF and unsure which quant? Q4_K_M, Q5_K_M, and Q8 trade quality for VRAM — and a smaller quant can beat a bigger one that spills to CPU. The full ladder, explained. (2026)
- 7 min read
llama.cpp --n-gpu-layers: -1, 0, Partial
Set --n-gpu-layers in llama.cpp with clear -1, 0, and partial offload rules, VRAM headroom checks, examples, and CPU fallback fixes.
- 7 min read
Local LLM on Consumer GPUs: 50 req/s, $0/Call [Benchmarks 2026]
Cloud LLM bills hit $2K/month fast. An RTX 5070 Ti serves Llama 3.1 at 50 req/s for $0 per call — we benchmarked 4 consumer GPUs and built the exact production setup.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.