Topic
Hardware
54 posts on hardware — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Why production AI is moving to open weights
I tested a local-first writing pipeline on an RTX 5090. Gemma 4 writes the draft, deterministic checks catch errors, and frontier QA decides what ships.
- 6 min read
My 8B Model Failed a 400-Word Task
Three Llama 3.1 8B runs missed a 400-word floor. Here is the verifier-driven route that moved long-form synthesis to Gemma 4 26B.
- 6 min read
I Rebuilt 5,383 Embeddings After a Dimension Change
I moved a vault search index from 1,536 to 768 dimensions. Here is why I rebuilt all 5,383 chunks instead of mixing vector spaces.
- 6 min read
Pin Your Local LLM Context Size Before You Build a Router
Changing context size can reload a local model before every request. A measured RTX 5090 sweep shows why context belongs in the routing key.
- 8 min read
I built a self-improving code model on one RTX 5090. Here is what actually worked.
Six pieces, one consumer GPU, no cloud. The honest results: some parts worked, some were flat, and one idea changed everything.
- 5 min read
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
- 5 min read
How I Make Local Model Runs Fail Safely On A 5090
A local model run should prove its safety path before it proves a score. Here is the small guardrail loop I use on my RTX 5090 for QLoRA starter work.
- 5 min read
How to Make a Local QLoRA Starter Fail Safely
A local QLoRA starter should prove data, GPU safety, metrics, tests, and blockers before it claims progress. Here is the small loop I use on owned hardware.
- 2 min read
VRAM Calculator: Estimate Local LLM Requirements
Estimate the VRAM required to run local LLMs like Llama 3 with our interactive calculator. Compare quantization levels like Q4 and Q8 to plan your hardware.
- 7 min read
llama.cpp Multi-GPU Guide: --tensor-split & --split-mode (2026)
Running a 70B across 2 GPUs and hitting OOM? The llama.cpp multi-GPU reference: --tensor-split ratios, --split-mode layer vs row, --main-gpu, and the VRAM math from a real two-card rig.
- 6 min read
Your AI, Your Rules: Engineering Agents for Digital Freedom
Recent events highlight the growing need for user control and autonomy in the digital world. Discover how engineering AI agents on your own hardware empowers true digital freedom, safeguarding your data and decisions against centralized forces.
- 6 min read
Q4_K_M vs Q5_K_M vs Q8: Which GGUF Quant?
Compare GGUF quants by actual file size, runtime memory, and task results. Pick a candidate, test your context, and keep the version that passes.
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.