A GPU Driver Is Not a Local LLM Benchmark
My 2026-08-20 RTX 5090 report lists driver 610.88 and 17 benchmark rows. That does not show the driver changed model speed.
My RTX 5090 report on 2026-08-20 named NVIDIA driver 610.88. It also carried 17 benchmark rows. I did not write that the driver made any local model faster.
Short answer: A driver version belongs in a local LLM benchmark receipt, but it is not a benchmark result. To claim that a driver changed model speed or task quality, I need a controlled before-and-after run through the same runtime path. Canonical URL: https://bmdpat.com/blog/gpu-driver-not-local-llm-benchmark-2026
Why is a GPU driver version not a benchmark result?
A version string tells me what was installed when I captured a result. It does not tell me what caused that result.
My 2026-08-20 vault notebook records NVIDIA driver 610.88 and 17 benchmark rows. The public 5090 Reports page is the same notebook series. On 2026-08-20 it still showed the 2026-07-09 artifact: driver 610.62 and 6 measured rows.
The 2026-08-20 notebook has useful data. It does not contain a matched pair that runs one workload before and after the driver change. Without that pair, I cannot separate a driver effect from a different model load, batch setting, context size, or work on the host.
This boundary matters because a local model stack has more than one moving part. The GPU driver sits below CUDA, the runtime, the model file, the quant, and the request. A new number after any one change is a new observation. It is not yet a cause.
What must stay fixed when I test a driver change?
I keep the production route fixed first. My measurements use Ollama's generate API, not a separate benchmark binary. The Ollama generate API is the route whose result matters to the sizing work.
For a useful pair, I keep the model tag, quant, Ollama version, API request, num_ctx, num_predict, temperature, and GPU placement fixed. I also keep the host quiet enough to record what else ran during each test.
Then I capture more than tokens per second. I record load time, prompt work, output rate, generated token count, GPU memory, power when I have it, and the task result. A driver test needs the same proof fields as any other local model test.
The VRAM fit and runtime support check covers a related trap. A model can fit in VRAM and still fail on the runtime route. A driver update can also leave the model route unchanged. Both facts belong in the receipt.
What does the 2026-08-20 report actually say?
The vault notebook says the RTX 5090 had driver 610.88 when the 2026-08-20 report ran. It says benchmark data was present. It lists 17 rows from its benchmark inputs. The public page is not that receipt.
That gives me a hardware snapshot and a source list. It does not give me a driver comparison. Some rows record an Ollama version and workload details. The report does not mark a row as the same run from before this driver version.
I keep that difference clear in public notes. The driver value is context. The benchmark row is evidence for only the route, settings, and task that the row names.
How do I make a driver claim that holds up?
I start with an old receipt that has the full settings. I change the driver and nothing else that affects the route. I rerun the same request enough times to see the spread. I keep the outputs and task checks beside the timing data.
Then I compare the pair. If the task result changes, I inspect the output before I call it better or worse. If the output rate moves but the host state differs, I call the result inconclusive. If the pair stays close, I say the test found no material change for that route.
This is the same discipline behind my benchmark provenance snapshot. A result stays useful when a later reader can see what changed and what stayed fixed.
What do I record after the next driver update?
I will add the driver version to the receipt. I will not make it the headline. The headline comes after a matched workload produces a measured difference that survives the task check.
This rule slows down one sentence. It saves time later. I do not have to retract a speed claim because the model had a warm cache, a different batch size, or a busy GPU.
Accompanying prompt
What the prompt does: checks whether a local LLM driver comparison has enough evidence for a speed or quality claim.
Copy/paste this prompt:
Copy-ready prompt
Paste the exact block into your coding agent.
No article chrome, no footnotes, no formatting drift.
This prompt and every other one we publish live in the free prompt library.
Copy the block above.
Get measured local AI notes by email: https://bmdpat.com/5090-reports
Get the Local AI Field Kit
Four copy-ready tools now, then one evidence-backed Local AI Lab Note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.
Patrick Hughes
Building BMD HODL — a one-person AI-operated holding company. Nashville, Tennessee. Twenty-Two agents.
More writing
- 5 min
The 26B Model Hit the Cap. The 8B Finished.
Same code task, same 512-token cap. Ollama's done_reason showed one local model finished and one got cut off. Tokens per second hid it.
- 6 min
How I Keep LLM Results Valid After a Driver Update
A new GPU snapshot does not refresh an old benchmark. I show how I bind driver, runtime, workload, and timestamps to each local LLM result.
- 5 min
When a 4B Local LLM Beats 26B on One Task
On one RTX 5090 workshop, a 4B model beat a 26B model on speed while both passed four code checks. Here is the model-selection rule I kept.
- 5 min
The faster local model run took 83x longer
One llama3.1:8b run posted 188.16 tok/s and finished in 43.9 seconds. The next posted 153.12 and finished in 0.53. Tokens per second ranked them backwards.
- 5 min
My 5090 benchmark was missing the field I needed most
A fresh Qwen3.5 9B run showed 84.94 tok/s, but the useful number was the 6,105 ms load phase. I added phase timings and capture time to the benchmark receipt.