Topic
Local LLM
84 posts on local llm — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Ollama Load Time Can Hide a Fast Local LLM
My RTX 5090 run spent 43.48 seconds on model load and 0.17 seconds on output. Split wall time before you judge local LLM speed.
- 5 min read
A 32 GB GPU Still Needs Host RAM Headroom
My RTX 5090 had 32 GB of VRAM ready, but my local policy stopped Ollama at 7.02 GiB of free host RAM. Here is my preflight before model load.
- 5 min read
Your Local LLM CSV Needs a Schema Version
I audited seven RTX 5090 benchmark CSVs and found four header widths, three power schemas, and no schema version. Here is the row contract I use now.
- 5 min read
Your Benchmark Row Never Saved the Driver Version
I audited seven RTX 5090 benchmark files on 2026-08-27. Not one records the GPU driver version. Here are the four fields every local LLM row needs.
- 5 min read
Preflight Local AI Before You Benchmark a Model
My local AI test stopped before model load because the runtime could not build. This preflight keeps setup failures out of benchmark results.
- 5 min read
49W Average Hid a 338W Burst on Gemma 26B
Gemma 4 26B Q4_K_M averaged 49 W on a long RTX 5090 run and peaked at 338 W. Keep both watt numbers before you compute energy per token.
- 5 min read
How to Calculate Local LLM Energy per Token
My RTX 5090 test shows how watts and output rate become joules per token, and why the faster of two matched settings can waste energy.
- 4 min read
Why a Failed Local LLM Benchmark Row Still Matters
A failed local LLM row marks the test boundary. My RTX 5090 report shows why quality, speed, and settings belong in one receipt.
- 5 min read
Mojo Is Open Source. What Local AI Builders Need to Know
Modular put Mojo under Apache 2.0 with LLVM exceptions on August 18, 2026. Here is what the release changes for local AI builders, and what it does not.
- 5 min read
My local LLM eval hid four token caps
My 5090-rig eval uses four output caps: 50, 180, 192, and 256 tokens. A score without the cap is not a model result. Record the cap on every row.
- 5 min read
A GPU Driver Is Not a Local LLM Benchmark
My 2026-08-20 RTX 5090 report lists driver 610.88 and 17 benchmark rows. That does not show the driver changed model speed.
- 5 min read
The 26B Model Hit the Cap. The 8B Finished.
Same code task, same 512-token cap. Ollama's done_reason showed one local model finished and one got cut off. Tokens per second hid it.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.