Field journal
Blog archive
Every published note, with the newest work first. Use the topic links to stay inside one search intent.
I tested four Ollama settings on Gemma 4 26B. num_batch 256 delivered nearly all the speed of 512 while using less power on my RTX 5090.
A 6.66 GiB ternary 27B model fit my RTX 5090, but it lost the default slot. File size is only the first local model-selection gate.
Agent logs show activity. They do not prove the outcome an agent claimed. Here is the verification layer I added after an agent reported two false dones.
I extracted the claims ledger that runs my agent operation into showwork. Falsifiable claims, deterministic checks, and an exit gate that refuses a false done.
Ollama's $65M round and 8.9 million developers make local AI a funded product category. Here is what changes for builders on owned GPUs.
The nightly queue sweep aborted for the fourth straight night. The root cause is a critical failure in the vault health check. I discovered a private path wo...
I tested a local-first writing pipeline on an RTX 5090. Gemma 4 writes the draft, deterministic checks catch errors, and frontier QA decides what ships.
Stuck between q4km and q5km? Verdict: start with Q4_K_M on 8–24GB cards; move to Q5_K_M only if it still fits with KV-cache headroom. The 5-minute VRAM check inside.
Three Llama 3.1 8B runs missed a 400-word floor. Here is the verifier-driven route that moved long-form synthesis to Gemma 4 26B.
Blog charts need correct labels. I tightened layout geometry, char budgets, claims checks, and vision logs so a bad PNG cannot ship quietly.
I moved a vault search index from 1,536 to 768 dimensions. Here is why I rebuilt all 5,383 chunks instead of mixing vector spaces.
Preview the files your local LLM will receive before inference. This simple gate catches bad retrieval while the fix is still cheap and clear.
Change num_ctx in Ollama and it reloads the whole model. On my RTX 5090 that meant 140 seconds of dead air per context swap. Here is how to stop paying it.
Changing context size can reload a local model before every request. A measured RTX 5090 sweep shows why context belongs in the routing key.
Claude and Codex both went dark. Here is the tertiary Gemini path I wired, with honest qa_reviewer stamps.
Six pieces, one consumer GPU, no cloud. The honest results: some parts worked, some were flat, and one idea changed everything.
A local LLM needs about half a gigabyte of VRAM per billion parameters at Q4, then KV cache and context stack on top. Here is how to know a model fits before you download 40 GB.
A bigger local model will not fix a stuck runtime. Add a bounded inference doctor first, then trust the benchmark.
Stop scoring your local model on how close it gets to Opus. It is a different tool with a different sweet spot. Here is the line, and which side your work sits on.
Local LLMs are useful when the loop proves the output, not when the benchmark looks good. This is the small gate I use before a local coding agent gets more rope.
A local model is not ready because it runs fast. It is ready when one verifier loop can prove the output before an agent writes files.
A local model run should prove its safety path before it proves a score. Here is the small guardrail loop I use on my RTX 5090 for QLoRA starter work.
A local QLoRA starter should prove data, GPU safety, metrics, tests, and blockers before it claims progress. Here is the small loop I use on owned hardware.
A local LLM workflow needs more than a model prompt. It needs a verifier loop that proves the file, command, URL, or report changed before the agent claims done.