Topic
Inference
3 posts on inference — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 7 min read
llama.cpp Multi-GPU: Splitting a Model Across Cards with --tensor-split
Split a 70B model across multiple GPUs with llama.cpp. How --tensor-split, --main-gpu, and --split-mode work on a real consumer rig.
- 6 min read
llama.cpp -ngl 99 Still on CPU? 5 Fixes, Ranked (2026)
You set -ngl 99 and llama.cpp still pins the CPU — the flag isn't the bug. Here's the 30-second load-log check and the 5 real causes, ranked by how often they bite.
- 7 min read
llama.cpp n-gpu-layers Explained: -1 vs 0 + VRAM Guide (2026)
Setting --n-gpu-layers wrong tanks your tokens/sec or crashes with OOM. Here's exactly what to use (-1, 0, or a number), the VRAM-per-layer math, and 4060-4090 benchmarks.
The AI agent build notes
Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.