Topic
Model Evaluation
3 posts on model evaluation — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
The faster local model run took 83x longer
One llama3.1:8b run posted 188.16 tok/s and finished in 43.9 seconds. The next posted 153.12 and finished in 0.53. Tokens per second ranked them backwards.
- 6 min read
Build Local LLM Eval Data From Real Failures
I show how I turn failed local coding runs into replayable eval rows with the prompt, model output, tests, route, and verifier result intact.
- 5 min read
Why I Did Not Promote My Smaller Local Model
My smaller local model existed and ran, but it did not beat the baseline. Here is the promotion gate I use before changing a working local AI route.
The AI agent build notes
Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.