Topic
Model Evaluation
7 posts on model evaluation — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
How to Calculate Local LLM Energy per Token
My RTX 5090 test shows how watts and output rate become joules per token, and why the faster of two matched settings can waste energy.
- 4 min read
Why a Failed Local LLM Benchmark Row Still Matters
A failed local LLM row marks the test boundary. My RTX 5090 report shows why quality, speed, and settings belong in one receipt.
- 5 min read
My Local LLM Got Faster After It Passed the Tests
A three-run RTX 5090 test showed why local LLM tuning must pair speed with fixed-task checks. One faster setting also repaired every test.
- 6 min read
My local models refused zero of 50 security tasks
Hosted coding assistants have declined defensive security work. I ran 50 such tasks across five local models on my own hardware and counted zero refusals.
- 5 min read
The faster local model run took 83x longer
One llama3.1:8b run posted 188.16 tok/s and finished in 43.9 seconds. The next posted 153.12 and finished in 0.53. Tokens per second ranked them backwards.
- 6 min read
Build Local LLM Eval Data From Real Failures
I show how I turn failed local coding runs into replayable eval rows with the prompt, model output, tests, route, and verifier result intact.
- 5 min read
Why I Did Not Promote My Smaller Local Model
My smaller local model existed and ran, but it did not beat the baseline. Here is the promotion gate I use before changing a working local AI route.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.