Topic
Model Evaluation
5 posts on model evaluation — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
My Local LLM Got Faster After It Passed the Tests
A three-run RTX 5090 test showed why local LLM tuning must pair speed with fixed-task checks. One faster setting also repaired every test.
- 6 min read
My local models refused zero of 50 security tasks
Hosted coding assistants have declined defensive security work. I ran 50 such tasks across five local models on my own hardware and counted zero refusals.
- 5 min read
The faster local model run took 83x longer
One llama3.1:8b run posted 188.16 tok/s and finished in 43.9 seconds. The next posted 153.12 and finished in 0.53. Tokens per second ranked them backwards.
- 6 min read
Build Local LLM Eval Data From Real Failures
I show how I turn failed local coding runs into replayable eval rows with the prompt, model output, tests, route, and verifier result intact.
- 5 min read
Why I Did Not Promote My Smaller Local Model
My smaller local model existed and ran, but it did not beat the baseline. Here is the promotion gate I use before changing a working local AI route.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.