Writing · Tag

#5090

1 post tagged #5090.

Jul 7, 20265 min read
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
#local-llm#vram#quantization#llama.cpp

The AI agent build notes

Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.