Topic
Ollama
22 posts on ollama — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Unload Local LLMs After Every Test
A local model test is not over when text appears. I unload the model, read idle VRAM, and record the result before I start another run.
- 5 min read
How I Benchmark Local LLMs Before I Trust Them
A local LLM benchmark should end with a decision. I record task quality, tokens per second, VRAM, power, and failure state before I add a model.
- 5 min read
How I Test a 30B Local Model Before I Load It
A 30B local model can fit on paper and still fail the job. This test plan checks memory, tool use, speed, and repeatability first.
- 5 min read
When a 4B Local LLM Beats 26B on One Task
On one RTX 5090 workshop, a 4B model beat a 26B model on speed while both passed four code checks. Here is the model-selection rule I kept.
- 5 min read
Does Ollama Include That New llama.cpp Feature?
A llama.cpp release note does not prove that Ollama can use the feature. I trace the active runtime path, pin versions, and test the same workload.
- 5 min read
My Local LLM Got Faster After It Passed the Tests
A three-run RTX 5090 test showed why local LLM tuning must pair speed with fixed-task checks. One faster setting also repaired every test.
- 5 min read
The faster local model run took 83x longer
One llama3.1:8b run posted 188.16 tok/s and finished in 43.9 seconds. The next posted 153.12 and finished in 0.53. Tokens per second ranked them backwards.
- 5 min read
Why Local LLM Benchmarks Need Power Data
A local LLM benchmark that reports only tokens per second hides the operating cost. My RTX 5090 run shows what power data adds.
- 5 min read
VRAM Fit Is Not Runtime Support
A local model can fit in VRAM, download cleanly, and still fail before the first token. My 5090 test adds runtime support as a separate gate.
- 5 min read
My 5090 benchmark was missing the field I needed most
A fresh Qwen3.5 9B run showed 84.94 tok/s, but the useful number was the 6,105 ms load phase. I added phase timings and capture time to the benchmark receipt.
- 6 min read
Search Old Results Before Publishing an LLM Test
An independent QA pass caught my second post about the same Ollama batch sweep. Here is the duplicate check I now run before publishing an LLM result.
- 6 min read
Why I Benchmark Local LLM Input and Output Separately
My RTX 5090 runs show why model loading, prompt ingestion, token generation, and task checks need separate measurements before production use.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.