Topic
Hardware
17 posts on hardware — guides and lab notes from real runs on hardware we own. New posts land here automatically. Start anywhere, or grab the copy-paste prompts that ship with them.
- 5 min read
Will That Local Model Fit? Do the VRAM Math First
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
- 5 min read
How I Make Local Model Runs Fail Safely On A 5090
A local model run should prove its safety path before it proves a score. Here is the small guardrail loop I use on my RTX 5090 for QLoRA starter work.
- 5 min read
How to Make a Local QLoRA Starter Fail Safely
A local QLoRA starter should prove data, GPU safety, metrics, tests, and blockers before it claims progress. Here is the small loop I use on owned hardware.
- 2 min read
VRAM Calculator: Estimate Local LLM Requirements
Estimate the VRAM required to run local LLMs like Llama 3 with our interactive calculator. Compare quantization levels like Q4 and Q8 to plan your hardware.
- 5 min read
GGUF Quant Cheat Sheet: Q4 vs Q5 vs Q6 vs Q8 (2026)
Skip the theory — a one-glance decision table for Q4_K_M, Q5_K_M, Q6_K, and Q8_0 on consumer GPUs, with the quality and size tradeoff spelled out for each.
- 5 min read
Tune llama.cpp --n-gpu-layers: VRAM Math & OOM Fixes (2026)
Set --n-gpu-layers too high and you OOM; too low and inference crawls. The VRAM math, KV-cache sizing, and a fast tuning loop to find the right value for your GPU. (2026)
- 5 min read
GGUF Quantization and VRAM: How to Pick Q4, Q5, or Q8 for Your GPU (2026)
VRAM decides your GGUF quant, not vibes. How I assign Q4, Q5, Q8 across an 8GB 3070, 16GB 5070 Ti, and 32GB 5090.
- 7 min read
llama.cpp Multi-GPU: Splitting a Model Across Cards with --tensor-split
Split a 70B model across multiple GPUs with llama.cpp. How --tensor-split, --main-gpu, and --split-mode work on a real consumer rig.
- 6 min read
How to Tune --n-gpu-layers for Your VRAM Budget
How to actually pick --n-gpu-layers: the offload math, finding the number with nvidia-smi, multi-GPU splits, and the top OOM mistakes.
- 6 min read
How to Pick a GGUF Quant Level for Your VRAM Budget
Given your GPU, which GGUF quant do you actually pick? The VRAM math, a card-by-card table, and the quality tradeoff in plain terms.
- 6 min read
Your AI, Your Rules: Engineering Agents for Digital Freedom
Recent events highlight the growing need for user control and autonomy in the digital world. Discover how engineering AI agents on your own hardware empowers true digital freedom, safeguarding your data and decisions against centralized forces.
- 8 min read
GGUF Quantization 2026: Q4_K_M vs Q5 vs Q8 — Which to Pick
Short answer: Q4_K_M wins for most local LLMs — 75% smaller with near-zero quality loss. Q5, Q6 and Q8 each win edge cases. Benchmarked on real GPUs — here's the pick for your VRAM. (2026)
- 5 min read
GPU Prices Up 48% in Two Months. I Run LLMs in My Garage.
Blackwell rental hit $4.08/hr. CoreWeave raised prices 20%. Anthropic restricted their newest model to 40 orgs. Meanwhile, consumer GPUs are sitting idle.
- 6 min read
Raspberry Pi 5 Offline Voice Assistant: Sub-2s, No Cloud (2026)
Want a private voice assistant with zero cloud and no subscription? A Raspberry Pi 5 runs it offline at sub-2s latency. We tested 6 local models on real hardware — here's the winner. (2026)
- 7 min read
Local LLM on Consumer GPUs: 50 req/s, $0/Call [Benchmarks 2026]
Cloud LLM bills hit $2K/month fast. An RTX 5070 Ti serves Llama 3.1 at 50 req/s for $0 per call — we benchmarked 4 consumer GPUs and built the exact production setup.
- 7 min read
OpenClaw vs Custom AI Agents: 3x Faster to Ship, 2x the Cost — Real Numbers Inside
We ran the same AI agent on OpenClaw and a custom build for 90 days. Shipping was faster — but the monthly bill, vendor lock-in, and control gaps tell a different story. Full breakdown with actual costs.
- 8 min read
How I Let an AI Agent Run 100 ML Experiments Overnight on a $500 GPU
I let an autonomous agent run 100 ML experiments while I slept. 7 succeeded. Net result: 25% model improvement. Here's the setup.
The AI agent build notes
Real costs, real tools, no fluff. M-F when I ship, publish, or learn something worth sending.