Topic
Local LLM
54 posts on local llm — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Your local LLM is not a worse Claude. It is a different tool.
Stop scoring your local model on how close it gets to Opus. It is a different tool with a different sweet spot. Here is the line, and which side your work sits on.
- 5 min read
A verifier loop beats a faster local model
Local LLMs are useful when the loop proves the output, not when the benchmark looks good. This is the small gate I use before a local coding agent gets more rope.
- 4 min read
How I Gate a Local Coding Model Before I Trust It
A local model is not ready because it runs fast. It is ready when one verifier loop can prove the output before an agent writes files.
- 5 min read
How I Make Local Model Runs Fail Safely On A 5090
A local model run should prove its safety path before it proves a score. Here is the small guardrail loop I use on my RTX 5090 for QLoRA starter work.
- 5 min read
How to Make a Local QLoRA Starter Fail Safely
A local QLoRA starter should prove data, GPU safety, metrics, tests, and blockers before it claims progress. Here is the small loop I use on owned hardware.
- 5 min read
How to Run Local LLM Verifier Loops on Owned Hardware
A local LLM workflow needs more than a model prompt. It needs a verifier loop that proves the file, command, URL, or report changed before the agent claims done.
- 2 min read
VRAM Calculator: Estimate Local LLM Requirements
Estimate the VRAM required to run local LLMs like Llama 3 with our interactive calculator. Compare quantization levels like Q4 and Q8 to plan your hardware.
- 7 min read
llama.cpp --tensor-split: Run a 70B Across 2 GPUs (2026)
Your 70B model won't fit on one GPU. Here's how llama.cpp --tensor-split, --main-gpu, and --split-mode spread it across two cards on a real consumer rig.
- 6 min read
llama.cpp -ngl 99 Still on CPU? 5 Fixes, Ranked (2026)
You set -ngl 99 and llama.cpp still pins the CPU — the flag isn't the bug. Here's the 30-second load-log check and the 5 real causes, ranked by how often they bite.
- 6 min read
The AI Whirlwind: Why Your Local Agent Matters More Than Ever
Amidst the big tech AI boom and new policy discussions, discover why building ethical, autonomous AI agents on consumer hardware is critical. Explore practical engineering insights and Python tips for true local control.
- 5 min read
Decoding the AI Summer: Building Accountable Agents for the User
As the AI world heats up, learn how to build AI agents that prioritize user control and transparency. Discover practical strategies for creating observable and accountable automation on your own hardware.
- 7 min read
Localmaxxing isn't theory. Here's what my 3-GPU rig actually does.
Tom Tunguz called it localmaxxing. I run a 3070 + 5070 Ti + 5090 in one box and serve Llama 3.1 8B locally every day. Here are the real tokens-per-second, the real watts, and the real cost per million tokens.
The AI agent build notes
Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.