I built a self-improving code model on one RTX 5090. Here is what actually worked.

I spent a night building a full local loop where a code model trains itself. It runs on one RTX 5090. No cloud model, no uploads, no human labels. The whole thing lives on localhost.

This is the honest write-up. Some parts worked. Some were flat. One idea made the rest possible.

The one idea that matters

The unlock is a verifier you trust.

For code, the verifier is easy: run the tests. If pytest passes, the code is correct. If it fails, it is not. That pass or fail is ground truth. No opinion, no reward model, no human in the loop.

Once you have that, a lot becomes possible. You can generate training data and keep only the rows that pass. You can score a model with a real number instead of a vibe. You can let an agent write code, run the tests, read the failure, and try again.

Everything below is built on that one thing.

The six pieces

I built it in phases, each a small program that reuses the one before it.

1. The verifier. Runs generated code plus its tests in a throwaway sandbox, returns a pass fraction.
2. The data factory. Takes coding tasks, generates solutions with the local model, runs the verifier, keeps the ones that pass. Full provenance on every row.
3. Fine-tuning. QLoRA on the local model using the verified data.
4. Reinforcement learning. Generate several answers per task, score each with the verifier, push the model toward the higher-scoring ones.
5. The agent. Give the model a task. It writes code, runs the tests, reads the failure, and fixes it. Loop until the tests pass.
6. The self-improvement loop. The agent solves tasks. Its verified solutions become new training data. Retrain. Measure. Repeat.

Base model was Llama 3.1 8B in 4-bit. Training ran in WSL on the 5090.

What worked

Fine-tuning on diverse data. This was the real win. On held-out tasks the model never trained on, fine-tuning lifted the pass rate by about nine points. Tasks like binary search went from 33 percent to 100 percent. The model learned actual algorithms it could not do before.

There was one catch that cost me hours, so learn from it. My first dataset was 150 rows of only three tasks. The model memorized them and got worse on everything else. The fix was diversity: I grew the task set to 144 distinct problems across strings, lists, math, search, recursion, and parsing. Diversity was the lever, not any training trick.

The agent loop. This one is fun. I ran the agent on 20 hard tasks the base model tends to fail on the first try. One-shot generation solved 7 of them. The agent, allowed to read the test failure and revise, solved 14. It doubled the solve rate by doing what a person does: run it, see the error, fix it.

That is the whole pitch for agents in one number. Same model, same tasks. The difference was letting it check its own work and try again.

What did not work

I am going to be straight about this part, because most posts are not.

Scaling the reinforcement learning backfired. More RL steps pushed the training score up but made the held-out score drop. The model drifted away from the good weights and broke tasks it used to pass. Adding a penalty to keep it anchored helped a little but did not fix it at the scale I ran. RL is real, but it is not a free win.

One pass of the self-improvement loop was flat. The loop ran end to end. The agent harvested 32 verified solutions, they passed a re-check before entering training, the model retrained. The held-out score did not move. The reason is honest and boring: the new solutions were alternate answers to tasks already covered, and it was a single pass. Real self-improvement needs the loop to reach tasks outside the current set and to run many rounds. That is a scale problem, not a broken mechanism.

The takeaway

The mechanics all work. The gains were real where the signal was strong, and flat where the scale was small. If I had to keep one sentence: a verifier you trust changes what you can do with a small local model, because it lets the model generate and check its own training data with no human labels.

That is the same reason I care about guardrails on any agent that acts on its own. If your agent is going to write code, run it, and feed the results back into itself, you want hard limits on what it can spend and do while it loops.

Accompanying prompt

What the prompt does: Has a local coding model generate solutions, verify them with pytest, and reuse the passing rows to fine-tune itself in a repeating loop.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Copy promptSelect text

Role: You are a local LLM practitioner running a code model on one owned GPU. Context: You have a local model (for example Llama 3.1 8B in 4-bit), a set of coding tasks, and pytest in a sandbox. No cloud APIs, no human labels. Task: Design a self-improvement loop with these stages: 1. Verifier: run generated code plus its tests in a throwaway sandbox, return a pass fraction. Pass or fail is ground truth. 2. Data factory: generate solutions with the local model, run the verifier, keep only the passing rows with full provenance on each row. 3. Fine-tune: QLoRA on the verified rows. 4. Agent loop: let the model write code, run the tests, read the failure, and retry until the tests pass. 5. Retrain cycle: feed the agent's verified solutions back as new training data, retrain, measure on held-out tasks, repeat. Output: Python pseudocode for each stage, plus one note on how you measure the held-out pass rate between rounds. Constraints: Everything runs locally. Keep a diverse task set so the model does not memorize a few tasks. Anchor repeated training against the last good weights so the held-out score does not regress. Cap what the loop can spend and do while it runs.

23 lines/1198 chars

Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

That is what I build AgentGuard for. It is a runtime budget, token, and rate limiter for AI agents. If you are running loops like the ones above, start here: AgentGuard.