Llama 3.1 8B
Measured at 73.74 tok/s with 5.4 GB peak VRAM.
ollama run llama3.1:8bMeasured GPU guide / RTX 3070
RTX 3070 local LLMs: what actually fits.
An 8GB RTX 3070 can run useful local models without CPU offload. These are owned-hardware measurements at Q4_K_M and 4K context, not VRAM estimates.
8GB GDDR6Q4_K_M4K contextMeasured 2026-07-26
Check and save my RTX 3070 rigMeasured results
Two models stayed fully on the GPU.
Both runs used Ollama and a code-generation workload. Generation speed, prompt speed, and peak VRAM come directly from the recorded run.
| Model | Generation | Prompt | Peak VRAM | Fit |
|---|---|---|---|---|
| Llama 3.1 8BQ4_K_M / 4K | 73.74 tok/s | 625.23 tok/s | 5.4 GB | Full GPU |
| Qwen3.5 9BQ4_K_M / 4K | 59.39 tok/s | 18.70 tok/s | 6.8 GB | Full GPU |
Qwen3.5 9B
Measured at 59.39 tok/s with 6.8 GB peak VRAM.
ollama run qwen3.5:9bContext matters
8GB is a real boundary.
The measured rows above are exact only at 4K context. A larger context cache consumes more VRAM and can move model layers to the CPU. That reduces speed. The sizing desk recomputes the answer for the context and workload you choose.
Size my exact RTX 3070 setupRTX 3070 FAQ
- Can an RTX 3070 run local LLMs?
- Yes. We measured Llama 3.1 8B Q4_K_M at 73.74 tokens per second and Qwen3.5 9B Q4_K_M at 59.39 tokens per second on an 8GB RTX 3070 at 4K context.
- What size LLM fits in 8GB of RTX 3070 VRAM?
- Our measured 4K-context runs kept quantized 8B and 9B models fully resident on the GPU. Exact fit changes with model architecture, quantization, and context length.
- Which quantization should I use on an RTX 3070?
- Q4_K_M is a practical starting point for the measured 8B and 9B models. It preserved full GPU residency in our 4K-context runs while leaving room for the context cache.
- Does a larger context window still fit?
- Larger context windows use more VRAM and can push layers onto the CPU. Use the sizing desk to check the specific model, quant, and context before downloading it.