VRAM Fit Is Not Runtime Support

A model can fit in your GPU memory and still fail before it produces one token.

That happened in my latest local model test. A 2.7 GB Nanbeige4.2-3B GGUF downloaded and registered on an RTX 5090 with 32 GB of VRAM. Ollama then stopped at model load with an unknown architecture error. The memory check passed. The runtime check failed.

This is why I now treat VRAM fit and runtime support as separate gates in the local model sizing workflow.

What did the 5090 test actually prove?

The test proved one narrow thing: Nanbeige4.2-3B was not runnable on this Ollama build. The Q4_K_M file downloaded. The model appeared in the local model list. The request failed with unknown model architecture: 'nanbeige' when Ollama tried to start its llama-server process.

I repeated the check with Q5_K_M. It failed at the same stage. That matters because it rules out a single quant file as the first suspect. The report records a runtime support problem, not a throughput result.

The GPU had 32 GB of total memory. The Q4_K_M model occupied 2.7 GB on disk. Those numbers make the model look like an easy fit. They do not tell us whether the serving stack can load its architecture.

Why is VRAM fit only the first gate?

I use four gates for a local model:

1. Artifact gate. Can I download and identify the model file?
2. Memory gate. Does the model fit the available VRAM, RAM, or offload plan?
3. Runtime gate. Does the named Ollama or llama.cpp version recognize the model architecture and quant format?
4. Workload gate. Does it complete the real task with acceptable quality, latency, and power?

The first two gates are easy to check from a model page and a GPU spec. The third requires a real load attempt. The fourth requires a real task. A sizing tool that stops after memory fit can turn an impossible download decision into a confident recommendation.

My VRAM fit guide still has a useful place. It answers the memory question. My honest local model evaluation notes answer the workload question. The missing middle is a named runtime check.

What should a runtime-support check record?

The check does not need a large benchmark suite. It needs a small receipt with enough detail to reproduce the result:

• model name and exact file or tag
• quantization and file size
• runtime name and version
• GPU name and available memory
• load result and exact error text
• one short generation only after load succeeds

The final item is important. Do not report tokens per second for a model that never reached generation. The correct result is load_failed, with the reason attached.

That is how the Nanbeige test is recorded. It produced no throughput number. Publishing a number from another machine would answer a different question and weaken the desk's data.

What about models that are too large to fit?

The same report tested Laguna S2.1 as a separate case. Its BF16 files total roughly 250 GB, and its smaller MXFP4 MoE build is split across three shards. That is not a near miss on a 32 GB card. Measuring it requires an NVMe expert-offload path, and that path did not build on this Windows host because the C toolchain was absent and WSL was unavailable.

That is another useful failure boundary. The report has no Laguna throughput number because the engine did not build and no inference ran. The next action is infrastructure work, not a guessed performance estimate.

How does this change model selection?

The sizing desk should show runtime support beside VRAM fit. A model recommendation should name the runtime version that loaded it, or mark support as unverified.

My current known-good rows show why this distinction helps. The 2026-08-02 5090 report includes llama3.1:8b rows for an agent code task and a long-context summary. They are workload rows from a model that loaded. They are useful only because the earlier gates cleared.

The practical order is simple: fit the model, load it, run the task, then compare speed. VRAM fit gets you to the starting line. Runtime support gets you a starting line at all.

Accompanying prompt

What the prompt does: It turns a local model check into a receipt that separates memory fit, runtime support, and workload evidence.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Copy promptSelect text

Role: local model operator checking a candidate for an owned GPU. Context: I have a model tag or GGUF file, GPU memory details, a runtime version, and a load or generation result. Task: 1. Check artifact identity and file size. 2. Check memory fit against the available VRAM and any stated offload plan. 3. Check whether the named runtime loaded the model architecture. 4. Record the exact load error when the model fails. 5. Report workload speed only after one real task completes. Output: a four-gate receipt with pass, fail, or unverified for each gate. Constraints: do not invent throughput. Keep another machine's result separate. Name the runtime version and preserve the exact error text.

10 lines/696 chars

Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

Get the artifact-backed local AI lab notes by email: https://bmdpat.com/5090-reports