---
date: 2026-07-09
generated_at: 2026-07-09 14:35
artifact_type: 5090-report
status: benchmark-live
gpu_name: "NVIDIA GeForce RTX 5090"
driver_version: "610.62"
memory_total_mib: "32607"
power_limit_w: "575.00"
benchmark_rows: 6
benchmark_failures: 0
source: nvidia-smi + Reports/5090/benchmarks/2026-07-09-5090-sweep-llama8b-gemma26b.csv
canonical: https://bmdpat.com/5090-reports
raw_markdown: https://bmdpat.com/5090-reports/latest.md
failure_artifact: https://bmdpat.com/5090-reports/failures/2026-06-12-gemma4-26b.md
---

# The 5090 Reports - 2026-07-09

Local-first AI experiments from controlled compute, starting with an RTX 5090 workstation. The artifact is the lab notebook.

## Verdict

First full measured sweep is live: six tokens/sec rows across two models and three workloads. The June 12 timeout is resolved and stays in the notebook.

## Method

- Runner: Ollama 0.31.1 via /api/generate, temperature 0, bounded num_predict.
- Tokens/sec comes from Ollama's own eval_count / eval_duration accounting.
- Watts and VRAM sampled from nvidia-smi every 0.5s during each run.
- Workloads: short generation (256 tok out), long-context summarize (~4.4k tok prompt), agent-style code task (512 tok out).
- One warmup request per model before measured runs.

## Hardware Snapshot

| GPU | Driver | Memory total MiB | Power limit W |
|---|---:|---:|---:|
| NVIDIA GeForce RTX 5090 | 610.62 | 32607 | 575.00 |

## Benchmark Table

| Model | Quant | Engine | Workload | Input | Output | Gen tok/s | Prompt tok/s | Peak VRAM MiB | Watts avg/max |
|---|---|---|---|---:|---:|---:|---:|---:|---|
| llama3.1:8b | Q4_K_M | Ollama 0.31.1 | short-gen-256 | 46 | 256 | 228.87 | 679 | 7170 | 111/439 |
| llama3.1:8b | Q4_K_M | Ollama 0.31.1 | long-context-summarize | 4344 | 128 | 206.74 | 12109 | 7826 | 130/332 |
| llama3.1:8b | Q4_K_M | Ollama 0.31.1 | agent-code-task-512 | 64 | 459 | 227.79 | 1028 | 7826 | 187/463 |
| gemma4:26b | Q4_K_M | Ollama 0.31.1 | short-gen-256 | 53 | 256 | 198.81 | 15 (cold) | 19932 | 43/187 |
| gemma4:26b | Q4_K_M | Ollama 0.31.1 | long-context-summarize | 4470 | 128 | 180.18 | 6179 | 20233 | 49/338 |
| gemma4:26b | Q4_K_M | Ollama 0.31.1 | agent-code-task-512 | 76 | 512 | 207.41 | 197 | 20233 | 48/320 |

## What Broke / Gotchas

- Changing num_ctx between requests forced a full model reload every time. For gemma4:26b (17 GB) that is roughly 140 seconds per swap. Pin num_ctx in agent stacks or pay the reload on every context change.
- The first gemma4:26b prompt-eval reading (15 tok/s) is a cold-start artifact; the same model hit 6179 tok/s prompt eval on the next request.
- Watts averages for gemma runs are dragged down by load time; the max column is the honest generation-burst number.
- The June 12 failure (5s timeout on this same model) was a timeout budget problem, not a model problem: /5090-reports/failures/2026-06-12-gemma4-26b.md

## Cost Note

At 227.79 tok/s and 187 W average, a million output tokens on this card costs about 0.23 kWh of electricity. At $0.12/kWh that is roughly 3 cents per million output tokens, before amortizing the hardware. Local beats API economics only when the workload repeats enough; this is the measured input to that math.

## Distribution Contract

- Public posts must include one artifact: this report, a benchmark table, a diagram, or a repo note.
- CTA is the email list on bmdpat.com.
- One new experiment per week. It must produce a measured artifact or a failure log.

## Next Measurement

Run a real agent workload end to end through the local stack and capture latency per tool call plus API-equivalent cost per task, not just raw tokens/sec.