loop-engineering-skill

Why I built it

Loop Engineering sounds irresistible: give an agent a goal, let it inspect its own work, and keep going until it succeeds. The human gets out of the repetitive part.

But the expensive failure is not one wrong answer. It is a weak verifier calling the wrong direction “progress,” so the agent spends token after token polishing garbage into something that merely looks finished. A stronger model can make this worse: it gets better at satisfying the judge, including a judge that measures the wrong thing.

I build agent products by day and study agent evaluation at Tongji by night, and I keep returning to the same conclusion: the verifier decides whether the next token buys progress or more garbage. So this skill starts one step earlier than execution. Before an agent builds a loop, it asks whether the task should be looped at all.

A person stands outside a work loop and checks whether the work should be automated, while another person relaxes inside an unchecked loop — The skill does not work inside the loop. It stands outside first and asks whether the work is safe and measurable enough to hand off.

First, teach it to say “no”

Claude Code ships a bundled /loop that executes a loop — it re-runs a prompt at a fixed or adaptive interval until you stop it. This skill is the other half: it runs nothing. It helps you decide the two things /loop doesn’t — whether a task should be looped at all, and how to set the verifier, gates, and stop conditions. Use it to design the loop right, then point /loop, a cron job, or a queue worker at it.

What it decides before anything runs

Once it’s active, the AI stops treating “wrap it in a loop and let it run” as the default. It first does the thing most people skip — triage: a GREEN / YELLOW / RED verdict + decisive reasons across four dimensions (can a machine judge right from wrong, is judging it cheap, can a mistake be undone, can it be split small), with two hard vetoes — no faithful verifier, or an irreversible action that can’t be gated — going straight to RED.

GREEN: a faithful, cheap verifier, reversible, decomposable — a deterministic, largely unattended loop.
YELLOW: the verifier needs a model + rubric, or steps are only semi-reversible — keep a human checkpoint, cap iterations tightly.
RED: no faithful verifier can be built, or an irreversible action can’t be gated — don’t fully automate; build a verifier first, or keep a human in the loop.

Once a task clears triage, it designs the loop in three layers (outer scheduler / inner refinement loop / commit gate) and holds a few red lines that production loops fail on. It serves five modes: assess, design, diagnose, harden, implement.

Once the loop runs, where humans still matter

maker ≠ acceptance authority. The model doing the work can’t grade its own output — it grades too leniently; and independence of evidence matters more than swapping in another model.
Protect the control plane. A stuck agent’s favorite “fix” is to weaken the test, not fix the artifact — the skill forbids the maker from touching the acceptance contract, locked tests, budgets, or stop conditions.
Irreversible actions live in a commit gate. Mass-sends, releases, money transfers — anything you can’t claw back — live outside the inner loop, behind approve → preview → commit.
Three hard stops, none optional: threshold met, budget hit, no-progress — and “no-progress” must watch a real signal, not something an agent can dodge with meaningless cosmetic edits.

A person runs inside a loop bounded by a lock, representing verifier rules, budgets, and a commit gate — Safety is not about slowing the loop down. It is about keeping irreversible actions outside it: preview first, approve second, commit last.

Its temperament: an honest advisor that says “no”

Its design philosophy is one line: make the AI an honest advisor that will say “you can’t build a verifier for this, so don’t fully automate it” — not an “everything automates” salesperson.

Even if you explicitly ask for “full autonomy,” it won’t strip the approval / sandbox / commit gates — it gives you the closest design that stays inside a safe boundary. The things it should stop you on are exactly the ones where the judge can’t be built but you really want full autonomy — pure creativity, setting direction, calling strategy.

How it was evaluated, and where it stands

I ran a paired control/treatment battery: the same tasks answered by fresh agents without the skill and by agents that read it. The set covers five scenarios across opus / sonnet / haiku, plus held-out tasks from domains the skill never names. The finding is deliberately honest: a strong base model already refuses the obvious failures — moving money, deleting tests, deploying without approval. The skill’s marginal value is the consistency and verifier structure that stop less obvious loops from wasting tokens on false progress. That value should grow on smaller / faster models and across repeated runs.

The evaluation method itself is still something I’m iterating on — rigorously proving “the skill actually makes loops more reliable” is a hard problem, and exactly what my research is after. It’s v1; the repo open-sources the evals (cases, prompts, rubric, raw outputs, a limitations writeup) and schemas. Issues welcome — let’s iterate.

How to install & use

Install: clone the repo, copy the loop-engineering/ directory into ~/.claude/skills/, start a new session.
Use: it activates when you design / evaluate / diagnose / harden an agent loop; or just ask “is this a good fit for a loop?”
Read: SKILL.md is the core judgment; three references cover production hardening, deployment, and the full “why it’s reinforcement learning moved to inference time” reasoning.