Topic
Ollama
38 posts on ollama — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
VRAM Fit Is Not Runtime Support
A local model can fit in VRAM, download cleanly, and still fail before the first token. My 5090 test adds runtime support as a separate gate.
- 5 min read
My 5090 benchmark was missing the field I needed most
A fresh Qwen3.5 9B run showed 84.94 tok/s, but the useful number was the 6,105 ms load phase. I added phase timings and capture time to the benchmark receipt.
- 6 min read
Search Old Results Before Publishing an LLM Test
An independent QA pass caught my second post about the same Ollama batch sweep. Here is the duplicate check I now run before publishing an LLM result.
- 6 min read
Why I Benchmark Local LLM Input and Output Separately
My RTX 5090 runs show why model loading, prompt ingestion, token generation, and task checks need separate measurements before production use.
- 6 min read
How I Budget VRAM for Shared Local AI Workloads
My RTX 5090 measurements show how I budget VRAM for a local model, a real workload, and the GPU processes that must stay resident beside it.
- 5 min read
Why I Test 3 Workloads Before Sizing a Local LLM
One local LLM speed number hides the work behind it. My RTX 5090 sweep shows why short generation, long context, and code need separate rates.
- 5 min read
Ollama num_batch: 256 Was My RTX 5090 Sweet Spot
I tested four Ollama settings on Gemma 4 26B. num_batch 256 delivered nearly all the speed of 512 while using less power on my RTX 5090.
- 5 min read
Ollama Raised $65M. What Builders Get
Ollama's $65M round and 8.9 million developers make local AI a funded product category. Here is what changes for builders on owned GPUs.
- 6 min read
My 8B Model Failed a 400-Word Task
Three Llama 3.1 8B runs missed a 400-word floor. Here is the verifier-driven route that moved long-form synthesis to Gemma 4 26B.
- 6 min read
I Rebuilt 5,383 Embeddings After a Dimension Change
I moved a vault search index from 1,536 to 768 dimensions. Here is why I rebuilt all 5,383 chunks instead of mixing vector spaces.
- 6 min read
Pin Your Context Window or Pay the Reload Tax
Change num_ctx in Ollama and it reloads the whole model. On my RTX 5090 that meant 140 seconds of dead air per context swap. Here is how to stop paying it.
- 6 min read
Pin Your Local LLM Context Size Before You Build a Router
Changing context size can reload a local model before every request. A measured RTX 5090 sweep shows why context belongs in the routing key.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.