Llama 3.1 8B
Fastest measured result: 228.9 tok/s (short response, 4K context).
ollama run llama3.1:8bMeasured GPU guide / RTX 5090
RTX 5090 local LLM speed, measured by workload.
Six owned-hardware runs show what changes between short generation, coding, and long-context work. Every number below came from the same 32GB RTX 5090, not a VRAM estimate.
32GB GDDR7Q4_K_M4K + 8K contextMeasured 2026-07-09
Check and save my RTX 5090 rigMeasured results
Context and workload move the answer.
Generation rate is the conversational speed you feel. Prompt rate matters when ingesting code or long documents. Peak VRAM proves both measured models stayed fully on the GPU.
| Model | Workload | Context | Generation | Prompt | Peak VRAM |
|---|---|---|---|---|---|
| Llama 3.1 8B | Short response | 4K | 228.9 tok/s | 679 tok/s | 7.2 GB |
| Llama 3.1 8B | Coding task | 4K | 227.8 tok/s | 1,028 tok/s | 7.8 GB |
| Llama 3.1 8B | Long summary | 8K | 206.7 tok/s | 12,109 tok/s | 7.8 GB |
| Gemma 4 26B | Short response | 4K | 198.8 tok/s | 15 (cold) tok/s | 19.9 GB |
| Gemma 4 26B | Coding task | 4K | 207.4 tok/s | 197 tok/s | 20.2 GB |
| Gemma 4 26B | Long summary | 8K | 180.2 tok/s | 6,179 tok/s | 20.2 GB |
Gemma 4 26B
Fastest measured result: 207.4 tok/s (coding task, 4K context).
Measured from our internal corpus. No public Ollama tag is available.
Size the model you actually want to run.
A fast 8B benchmark does not answer whether your preferred 27B, 32B, or 70B model fits. The sizing desk ranks the current model catalog for your workload, marks CPU offload honestly, and gives you the command.
Size my exact RTX 5090 setupRTX 5090 FAQ
- How fast is an RTX 5090 for local LLMs?
- In our owned-hardware Ollama sweep, Llama 3.1 8B Q4_K_M generated 228.9 tokens per second on a short 4K run and 206.7 tokens per second on an 8K summarization run.
- What size LLM fits in 32GB of RTX 5090 VRAM?
- Our 26B Q4_K_M measurement stayed fully resident at both 4K and 8K context, peaking at 20.2GB. Exact fit still depends on architecture, quantization, and context.
- Can an RTX 5090 run a 70B model?
- A typical 70B Q4 model exceeds 32GB once weights and context are counted, so it usually needs CPU offload or more than one GPU. The sizing desk labels that boundary instead of calling it a full-GPU fit.
- Which RTX 5090 result should I use?
- Use the row that matches your workload and context. Generation speed, prompt processing, and VRAM pressure can change substantially between a short response, coding task, and long-context summary.