Writing · Tag

#inference

1 post tagged #inference.

Apr 6, 20267 min read
llama.cpp --n-gpu-layers Explained: -1, 0 & VRAM Calculator (2026)
Stop guessing your GPU layers. --n-gpu-layers -1 offloads everything to VRAM, 0 stays on CPU. See the exact VRAM-per-layer math, real 4060–4090 benchmarks, and find your optimal setting in seconds.
#llama-cpp#local-llm#gpu-offloading#n-gpu-layers

The AI agent build notes

Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.