Topic
Hardware
63 posts on hardware, guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 6 min read
Pin Your Local LLM Context Size Before You Build a Router
Changing context size can reload a local model before every request. A measured RTX 5090 sweep shows why context belongs in the routing key.
- 8 min read
I built a self-improving code model on one RTX 5090. Here is what actually worked.
Six pieces, one consumer GPU, no cloud. The honest results: some parts worked, some were flat, and one idea changed everything.
- 5 min read
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
- 5 min read
How I Make Local Model Runs Fail Safely On A 5090
A local model run should prove its safety path before it proves a score. Here is the small guardrail loop I use on my RTX 5090 for QLoRA starter work.
- 5 min read
How to Make a Local QLoRA Starter Fail Safely
A local QLoRA starter should prove data, GPU safety, metrics, tests, and blockers before it claims progress. Here is the small loop I use on owned hardware.
- 2 min read
VRAM Calculator: Estimate Local LLM Requirements
Estimate the VRAM required to run local LLMs like Llama 3 with our interactive calculator. Compare quantization levels like Q4 and Q8 to plan your hardware.
- 7 min read
llama.cpp Multi-GPU Guide: --tensor-split & --split-mode (2026)
Running a 70B across 2 GPUs and hitting OOM? The llama.cpp multi-GPU reference: --tensor-split ratios, --split-mode layer vs row, --main-gpu, and the VRAM math from a real two-card rig.
- 6 min read
Your AI, Your Rules: Engineering Agents for Digital Freedom
Recent events highlight the growing need for user control and autonomy in the digital world. Discover how engineering AI agents on your own hardware empowers true digital freedom, safeguarding your data and decisions against centralized forces.
- 6 min read
Q4_K_M vs Q5_K_M vs Q8_0 vs IQ4_XS: GGUF Quants Explained (2026)
Which GGUF file do you download? Start with Q4_K_M when VRAM is tight, test Q5_K_M or Q8_0 on your real task, keep the smallest that passes. What each quant means, IQ4_XS vs K-quants, a test to copy.
- 5 min read
GPU Prices Up 48% in Two Months. I Run LLMs in My Garage.
Blackwell rental hit $4.08/hr. CoreWeave raised prices 20%. Anthropic restricted their newest model to 40 orgs. Meanwhile, consumer GPUs are sitting idle.
- 7 min read
llama.cpp --n-gpu-layers: -1, 0, Partial GPU Offload (2026)
Not sure what to set --n-gpu-layers to? -1 offloads all layers, 0 keeps it on CPU, a number splits the model. VRAM headroom rules, examples, and the CPU-fallback fix. (2026)
- 6 min read
Raspberry Pi 5 Offline Voice Assistant: 6 Models Tested (2026)
Want a private voice assistant, zero cloud, no subscription? A Raspberry Pi 5 runs it offline at sub-2s latency. We tested 6 local models on real hardware; see the winner. (2026)
The AI agent build notes
Real costs, real tools, no fluff. One evidence-backed note on Friday when there is something worth sharing.
Get the requested artifact now, then at most one evidence-backed Local AI Lab Note on Friday when there is something worth sharing. One-click unsubscribe. No sponsored placements. Privacy.