# BMD PAT prompt pack 19 copy-paste prompts for local LLMs and AI agents, extracted from the bmdpat.com blog. Every prompt stays free on its source post; this file is the all-in-one copy. Live library (always current): https://bmdpat.com/prompts ## Local LLMs Need a Timeout Before They Need a Bigger Model **What it does:** It turns a local model launcher into a bounded inference doctor checklist. Source: https://bmdpat.com/blog/local-llm-timeout-guard-ollama-5090-2026 ```text Role: You are a local AI runtime reviewer. Context: I run local LLMs on owned hardware. The model server may be alive even when a generation probe hangs. Task: Design a small inference doctor for my local model stack. Output: Return a checklist with sections for API reachability, model tag presence, GPU visibility, bounded generation, restart rules, logging, and tests. Constraints: Use one tiny deterministic prompt. Use both a request timeout and a wall-clock timeout. Restart the model server only after bounded checks prove it is needed. Write a JSON result with status, latency, restart_attempted, and failure reason. Keep the plan small enough to implement in one script. ``` ## Your local LLM is not a worse Claude. It is a different tool. **What it does:** It helps you decide whether a task should run on a local model, a frontier API, or both. Source: https://bmdpat.com/blog/your-local-llm-is-a-different-tool ```text Role: You are a practical AI systems reviewer. Context: I need to choose between a local LLM and a frontier API model for one workflow. Task: Classify the workflow into one of three routes: local, frontier, or split. Output: 1. Recommended route. 2. Main reason. 3. Review gate needed before the result matters. 4. Failure mode to watch. 5. One test I should run before using it again. Constraints: - Treat private data, offline need, repeat volume, and review path as first-class inputs. - Do not choose local just because it is cheaper. - Do not choose frontier just because it is stronger. - Prefer the smallest model that can pass the review gate. ``` ## A verifier loop beats a faster local model **What it does:** This prompt turns a local model test into a measured verifier loop before the workflow gets more permission. Source: https://bmdpat.com/blog/local-model-verifier-loop-2026 ```text Role: You are a local AI agent evaluator. Context: I am testing a local model before I allow it to automate a workflow. Task: Design a verifier loop for this workload. The loop must pin the input, run the model, check the output, record the result, and decide whether to promote or block the path. Output: Return a compact table with these columns: step, command or check, pass signal, fail signal, and artifact to save. Constraints: Do not assume cloud APIs. Do not invent benchmark numbers. Mark any unmeasured value as unknown. Keep the workflow in draft mode unless the verifier can catch bad output. ``` ## How I Gate a Local Coding Model Before I Trust It **What it does:** Builds a small pass/fail gate for one local model task before you let it touch real files. Source: https://bmdpat.com/blog/local-model-verifier-loop-owned-gpu-2026 ```text Role: You are a local-model test engineer. Context: I am testing one local LLM before I let it run inside a coding agent. Task: Design a verifier loop for one task type. Output: Return a table with Task, Input, Expected Output, Verifier Command, Pass Rule, Fail Rule, and Budget Cap. Constraints: Use local files only. No network calls. No secrets. One command must decide pass or fail. Keep the test small enough to run after every model change. ``` ## How I Make Local Model Runs Fail Safely On A 5090 **What it does:** It turns a local model experiment into a fail-safe run contract before you spend a night on it. Source: https://bmdpat.com/blog/local-model-runs-fail-safely-5090 ```text Role: You are a local AI run reviewer for an owned GPU workstation. Context: I am about to run a local model job inside an agent loop. I need the job to fail safely before I trust any model score. Task: Review my run plan and list the missing safety gates. Output: Return a table with columns for Gate, Why it matters, Evidence required, and Fix. Constraints: Tie every number to a log, config file, or command output. Require a small first run. Require GPU memory and temperature checks before model load. Require cleanup in a finally path. Require append-only metrics. Require AgentGuard or an equal budget and rate limit for any agent loop. Do not accept hardcoded test data as proof. ``` ## How to Make a Local QLoRA Starter Fail Safely **What it does:** It turns a local fine-tuning idea into a small fail-safe starter with proof, resource checks, and honest blockers. Source: https://bmdpat.com/blog/fail-safe-local-qlora-starter-owned-hardware-2026 ```text Role: You are designing a fail-safe local QLoRA starter for owned hardware. Context: Hardware: [GPU, VRAM, cooling notes] Dataset: [path, row count, provenance, privacy limits] Current goal: [what the first run should prove] Task: 1. Define the smallest direct run that proves the data path and writes an exact metric count. 2. Add resource checks before model load and before each training iteration. 3. Define the fallback behavior if model load or training fails. 4. Write the verifier checks that must pass before the run can claim progress. 5. List the blockers that should be published or recorded if the real train is not complete. Output: - A one-command starter run. - A metrics schema with required fields. - A test checklist tied to the claim. - A blocker list with owner, next action, and proof needed. Constraints: - Keep the first run small. - Use exact file paths and row counts when available. - Label any stub or fallback metric clearly. - Do not claim model improvement without a verifier result. ``` ## How to Run Local LLM Verifier Loops on Owned Hardware **What it does:** It turns a local LLM task into a verifier loop with proof, retry limits, and AgentGuard runtime rails. Source: https://bmdpat.com/blog/local-llm-verifier-loops-owned-hardware-2026 ```text Role: You are designing a local LLM verifier loop for a single agent task. Context: The model runs on owned hardware. The task may touch private files, local repos, or local reports. The result is not accepted until an independent proof step passes. Task: Design the smallest useful verifier loop for this workflow: [paste workflow, target files, model, and command surface] Output: 1. Task scope: one job the local model is allowed to do. 2. Inputs: files, prompts, and commands the model can read. 3. Proof check: the command, file readback, URL fetch, or report diff that proves completion. 4. Retry rule: what to do after the first failed proof and when to stop. 5. Runtime limits: AgentGuard budget, token, rate, and retry ceilings. 6. Blocked packet: the exact note to write when the loop cannot pass. Constraints: - Keep the loop small enough to run today. - Do not trust model confidence as proof. - Prefer deterministic checks over judgment. - Do not expose secrets or private data to hosted tools. - End with either a passed proof or a written block. ``` ## Use Owner Gates and AgentGuard to Keep AI Agents Moving **What it does:** It turns a vague agent workflow into owner gates, AgentGuard cost rails, and proof checks. Source: https://bmdpat.com/blog/ai-agent-owner-gates-async-ops-2026 ```text You are helping me design owner gates and AgentGuard cost rails for an AI agent workflow. Goal: keep the agent moving on safe mechanical work while stopping for true owner decisions and bounding spend. Review this workflow: [paste workflow, agent prompt, or task list] Return: 1. Owner gates: decisions the agent must stop for. 2. Safe default actions: work the agent can do without asking. 3. Request packet: the exact fields the agent should write when blocked. 4. AgentGuard limits: budget, token, and rate ceilings for the run. 5. Proof checks: commands, files, URLs, or fields that prove completion. 6. Red lines: actions the agent must never take. Constraints: - Keep the policy short. - Prefer files, commands, and runtime checks over trust. - Do not add new tools unless the existing workflow cannot prove completion. - Use plain language a builder can maintain later. ``` ## AI Agent Memory: What Actually Works in 2026 **What it does:** Audits an agent workflow and turns vague memory into plain-file state, append-only events, and checks that prove each completion claim. Source: https://bmdpat.com/blog/ai-agent-memory-what-actually-works-2026 ```text You are auditing the memory layer for an AI agent workflow. Your job is to replace vague "the agent remembers" claims with durable state that a human can inspect and a script can verify. Work in five passes: 1. List every place the agent currently stores state. Separate durable files, append-only ledgers, databases, caches, prompts, and scratch output. 2. For each surface, answer: who writes it, who reads it, what proves it is current, and what happens if two agents write at once. 3. Find any completion claim that is not backed by a file, ledger row, API readback, test result, or live URL. 4. Propose the smallest durable memory system that would survive a cold restart: plain files first, append-only events second, database rows only where they are already part of the product path. 5. Write the exact checks that should run before an agent is allowed to say "done." Return: - A table of current memory surfaces and their risk. - A short list of state that should be deleted, archived, or ignored. - The proposed durable memory contract. - The verification commands or scripts that prove the contract. - One example "done" claim rewritten as a falsifiable claim. Constraints: - Prefer files a senior engineer can open and read. - Do not add a vector database unless the workflow has a specific retrieval failure that plain files cannot solve. - Do not trust model output as evidence. - Every important claim needs a deterministic readback. ``` ## GGUF Quant Cheat Sheet: Q4 vs Q5 vs Q6 vs Q8 (2026) **What it does:** Gives a fast, VRAM-keyed pick among Q4, Q5, Q6, and Q8 for one specific GPU and model. Source: https://bmdpat.com/blog/gguf-quant-which-to-pick-2026 ```text Role: You are giving a fast GGUF quant pick, not a lecture. Inputs: - GPU and VRAM: [e.g. 24GB 4090] - Model: [e.g. 32B] - Context length: [e.g. 8k] - One priority: [quality / speed / fit] Task: 1. In one line, tell me which quant to run: Q4_K_M, Q5_K_M, Q6_K, or Q8_0. 2. Give the VRAM it needs including KV cache at my context. 3. Give the one-line reason and what I trade off versus the next step up. 4. Only if it does not fit, name the fallback: smaller model or shorter context. Constraints: - Lead with the answer, then the numbers. - Account for context, not just weights. - Do not hedge across all four; pick one. ``` ## Tune llama.cpp --n-gpu-layers: VRAM Math & OOM Fixes (2026) **What it does:** Computes the max --n-gpu-layers that fits your VRAM and walks an OOM fix step by step. Source: https://bmdpat.com/blog/llama-cpp-n-gpu-layers-tuning-guide-2026 ```text Role: You are fixing an out-of-memory or under-offloaded llama.cpp run. My setup: - GPU and VRAM: [e.g. 12GB 3060] - Model and quant: [e.g. 14B Q4_K_M] - Context length: [e.g. 8k] - Current command: [paste your llama.cpp flags] - Symptom: [OOM at load / OOM mid-run / falls back to CPU / too slow] Task: 1. Estimate VRAM used by weights, KV cache at my context, and overhead. 2. Compute the highest --n-gpu-layers that fits, with headroom. 3. If it still will not fit, rank the fixes: lower quant, shorter context, smaller model, split. 4. Give the corrected command line. 5. Tell me exactly which load-log lines confirm the fix worked. Constraints: - Account for KV cache growth with context length. - Prefer a change that runs now over a hardware upgrade. - Do not silently drop layers to CPU without telling me the speed cost. ``` ## llama.cpp -ngl 99 Still on CPU? 5 Fixes, Ranked (2026) **What it does:** Diagnoses why llama.cpp still runs on the CPU after you set -ngl 99, using a load-log checklist. Source: https://bmdpat.com/blog/llama-cpp-ngl-cpu-fallback-fix-2026 ```text Role: You are diagnosing why llama.cpp runs on the CPU despite -ngl 99. My setup: - GPU, VRAM, driver: [e.g. 8GB 3070, driver 55x] - Build: [CUDA / Metal / ROCm / CPU-only?] - Model and quant: [e.g. 8B Q4_K_M] - Command: [paste flags] - What the load log shows: [paste the offload and backend lines] Task: 1. Confirm from the log whether llama.cpp was even built with GPU support. 2. Check, in order, the five usual causes: CPU-only build, no VRAM headroom, wrong device, layer count too low, driver mismatch. 3. Tell me which cause my log points to and the one command that proves it. 4. Give the corrected build or command. 5. Show the exact log line that will confirm the GPU is now in use. Constraints: - Read the load log before guessing. - Do not assume the flag is the problem; usually it is the build or VRAM. - Give one fix at a time with a check after each. ``` ## GGUF Quantization 2026: Q4_K_M vs Q5 vs Q8 — Which to Pick **What it does:** Recommends the GGUF quant level that best fits your VRAM, model, and quality tolerance. Source: https://bmdpat.com/blog/gguf-quantization-q4-q5-q8-explained-2026 ```text Role: You are choosing a GGUF quantization level for local inference. My setup: - GPU and VRAM: [e.g. 16GB 5070 Ti] - Model: [e.g. Qwen2.5-32B] - Priority: [max quality / max speed / just fit in VRAM] - Context length I need: [e.g. 8k] Task: 1. Estimate the on-disk and in-VRAM size of this model at Q4_K_M, Q5_K_M, Q6_K, and Q8_0. 2. Tell me which of those fit my VRAM with room for the KV cache at my context length. 3. Recommend one quant and say what I give up versus the next step up. 4. Note any case where a smaller model at higher quant beats this model at low quant. Constraints: - Be specific about numbers, not vibes. - Account for context and KV cache, not just weights. - Do not recommend a quant that will not load on my card. ``` ## Meta Burned 60T Tokens: Cap Your AI Agent Budget in 3 Steps **What it does:** Designs hard budget, token, and rate caps around an agent so a runaway loop cannot drain your account. Source: https://bmdpat.com/blog/meta-60t-token-burn-ai-agent-budget-control-2026 ```text Role: You are adding spend controls to an AI agent before it runs unattended. My setup: - Agent and model: [e.g. Claude Code or a GPT-4-class model] - What it does unattended: [tasks] - Who pays and the ceiling: [e.g. 50/day hard cap] - Current controls: [none / soft warnings] Task: 1. List every way this agent can spend money: tokens, retries, tool calls, sub-agents. 2. Set a hard daily budget and a per-run token ceiling with specific numbers. 3. Add a retry ceiling and a rate limit so one loop cannot spiral. 4. Define the kill condition: what trips it and what happens when it does. 5. Give me the check that runs before each agent run to enforce the caps. Constraints: - Caps must be hard stops, not logged warnings. - Assume the agent will hit an infinite loop at least once. - Prefer controls outside the model that the model cannot override. ``` ## Aymo AI Review 2026: Is $39/mo Worth It? (Free vs Paid) **What it does:** Runs a free-versus-paid decision on any AI subscription using your real usage, not the vendor pitch. Source: https://bmdpat.com/blog/aymo-ai-pricing-plans-review-2026 ```text Role: You are helping me decide whether a paid AI tool tier is worth it. The tool: - Name and tiers: [e.g. Aymo free vs 39/mo] - What the paid tier adds: [features, higher limits] My usage: - What I actually use it for: [tasks] - How often: [per day or week] - The limits I hit on free: [rate caps, missing features, none yet] Task: 1. List which paid features map to a real limit I actually hit. 2. Estimate my monthly value from the paid tier in hours or dollars saved. 3. Compare that to the price and to one cheaper or free alternative. 4. Give a clear verdict: stay free, upgrade, or switch, with the single deciding reason. Constraints: - Ignore features I will not use. - Do not accept marketing claims without a usage-based reason. - If the data is thin, say what to measure for two weeks before deciding. ``` ## llama.cpp n-gpu-layers Explained: -1 vs 0 + VRAM Guide (2026) **What it does:** Works out the right --n-gpu-layers value for your exact GPU and model, and explains the -1 vs 0 vs N tradeoff. Source: https://bmdpat.com/blog/llama-cpp-n-gpu-layers-explained-2026 ```text Role: You are tuning llama.cpp offload for a single machine. Context: I want to run a GGUF model with llama.cpp and set --n-gpu-layers correctly. My setup: - GPU and VRAM: [e.g. RTX 4090, 24GB] - Model and quant: [e.g. Llama-3.1-70B Q4_K_M] - Other VRAM users: [display, other models, none] Task: 1. Estimate how much VRAM one layer of this model costs at this quant. 2. Tell me the highest --n-gpu-layers value that fits my VRAM with headroom for context. 3. Explain what -1 and 0 do here and why I would pick a specific N instead. 4. Give the exact llama.cpp command line to start with. 5. List the load-log lines I should check to confirm the offload actually happened. Constraints: - Leave headroom for the KV cache at my context length. - Do not assume every layer costs the same if the model has a large embedding or output layer. - Prefer a value that runs today over a theoretical maximum. ``` ## Raspberry Pi 5 Offline Voice Assistant: Sub-2s, No Cloud (2026) **What it does:** Designs an offline voice assistant that runs on a Raspberry Pi 5, within a real latency budget. Source: https://bmdpat.com/blog/raspberry-pi-5-local-voice-ai-2026 ```text Role: You are designing an offline voice assistant on a Raspberry Pi 5. Constraints of my build: - Hardware: Raspberry Pi 5 [RAM: e.g. 8GB], [mic/speaker], no cloud. - Target: wake word to spoken reply under [e.g. 2 seconds]. - Everything runs locally, no internet calls. Task: 1. Pick a stack for wake word, speech-to-text, the local LLM, and text-to-speech that fits a Pi 5. 2. Give a rough latency budget per stage that adds up under my target. 3. Name specific model sizes and quant levels that run on this hardware. 4. Flag the stage most likely to blow the latency budget and how to cut it. 5. List the first three things to build and test, in order. Constraints: - Prefer components that install and run on ARM today. - Do not assume a GPU. - Keep memory use inside the Pi RAM budget above. ``` ## Local LLM on Consumer GPUs: 50 req/s, $0/Call [Benchmarks 2026] **What it does:** Designs a production local-LLM inference setup on consumer GPUs for real throughput. Source: https://bmdpat.com/blog/local-llm-inference-consumer-gpu-production-2026 ```text Role: You are designing a local LLM inference service on consumer GPUs. My setup: - GPUs and VRAM: [e.g. 2x 4090] - Model and quant: [e.g. 70B Q4_K_M] - Target load: [e.g. 50 requests/sec, prompt and response sizes] - Serving stack I can use: [llama.cpp, vLLM, other] Task: 1. Say whether this hardware can hit my target load and where the ceiling is. 2. Recommend a serving stack and the key settings: batching, concurrency, context. 3. Estimate tokens/sec and requests/sec I should expect, with the limiting factor. 4. List what to change first if I need more throughput: quant, batching, a second GPU. 5. Give the metrics to watch in production and their warning thresholds. Constraints: - Base numbers on the specific GPU and quant above. - Separate prompt processing from token generation limits. - Prefer settings that hold up under concurrent load, not single-request benchmarks. ``` ## AI Agent Pricing 2026: Real Cost from $500 DIY to $150K [Guide] **What it does:** Estimates the true build and run cost of an AI agent for your use case, by complexity tier. Source: https://bmdpat.com/blog/ai-agent-cost-pricing-2026 ```text Role: You are estimating the real cost of building and running an AI agent. My use case: - What the agent should do: [tasks] - Volume: [runs per day, users] - Build path: [DIY / contractor / platform] - Must-haves: [integrations, uptime, compliance] Task: 1. Place this in a tier: DIY weekend, small custom build, or enterprise rollout. 2. Break down build cost: engineering hours, tools, one-time setup. 3. Break down run cost: model tokens, infra, monitoring, maintenance per month. 4. List the hidden costs that vendor quotes usually leave out. 5. Give a low, likely, and high total, and the biggest cost driver to control. Constraints: - Separate one-time build from recurring run cost. - Treat token spend as variable and show how volume moves it. - Do not accept a single quote without a range. ``` --- Prompts are re-tested as models and tooling change. Get updates: https://bmdpat.com/newsletter