Why Agent Interfaces Run Backwards

There’s a nine-year-old I know who built a puzzle tower-defense game with Claude Code. It’s called Panther Defense. It has art, it has levels, it has a real storyline. He even turned his classmates into characters, one by one, and drew each of them by hand. The game actually works, and the link is right there: panther-defense.zeroliu.com.

Three years ago this would have been science fiction. Back then, even the kids people called “computer prodigies” couldn’t have pulled this off at eight or nine. When I first saw the finished thing, I just sat there, stunned.

What let him do it was, of course, the model. The model is now smart enough that a nine-year-old can build a complete game with it. That intelligence is real, and it’s the starting point.

But intelligence doesn’t automatically land in everyone’s hands. He can’t read a single line of code. What actually delivered that intelligence to him was the interface. He could see what the agent was doing: “I’m drawing the map.” “There’s an error here.” “Next I’ll add an enemy.” Because he could see it, he could correct it. Because he could correct it, he could keep going. How strong the model is, is one thing. Whether a nine-year-old can put that strength to work is another, and the interface is what decides it.

This kid is the living example of what I think of as an eager beginner with a blank slate: full of ideas, full of drive, and zero technical skill. So the question worth chewing on isn’t “is the model smart enough” — it already is — but rather: what kind of interaction lets more of these beginners steer agents that keep getting smarter?

Today’s agent interfaces are the last era’s playbook, run in reverse

I build AI products for a living, so I have some opinions about AI interaction. Over the past couple of years I’ve gone wide and pulled apart a bunch of today’s agent products — Codex, Claude Code, Cowork, Claude Artifacts, Google Stitch, Manus — taking the recurring interaction parts and laying them out side by side. And I noticed something interesting: this generation of design is, almost point for point, the last generation’s creed turned inside out.

The core creed of the last era’s software was hide the complexity. Black box, less friction, ideally you click once and the result appears, and whatever happened in between is something you don’t need to know and shouldn’t have to. The mark of a good product was that you couldn’t feel it working.

Agents do the opposite. They put the process on display, and at the critical moments they deliberately add friction. Streaming output, so you watch it speak one character at a time. A plan, a checklist of the steps it’s about to take, shown up front. Transparency throughout — which file it reads, which command it runs, which lines of code it changes, every action spread out in front of you. Before it changes anything, it shows you a diff, then sets up an approval gate and waits for you to click “allow” before it moves on. When something breaks, it doesn’t dump a stack trace on you; it tells you in plain language what went wrong and what you might do next. It can even replay the entire timeline.

Behind these parts there’s a pretty engineering-flavored truth. What the model emits isn’t “a block of text” at all — it’s a typed stream of events: text, thinking, tool calls, tool results, lined up in order. What the frontend does is essentially act as a component router: it dispatches each chunk to the matching component by type, then renders it incrementally as the tokens arrive. I went and read the source of an open-source agent, and sure enough there was a routing table just like this: text chunks go to the text component, tool chunks go to a tool card, thinking chunks go to a collapsible block. That smooth feeling you get inside a tool card, where the arguments seem to type themselves out, traces straight back to this chain of stream, route, render incrementally.

But the parts aren’t the point. Boil it all down to one sentence: today’s agent interaction is about laying out every step the AI takes, so a person can see it, stop it, and resume it. That’s what makes it so different from the last era.

Why do it this exact way

Just describing the status quo is boring. What’s actually worth thinking about is: why this, and not something else? My answer has two sides — one above the waterline, one below.

The user side: letting ordinary people correct course

A model running a long task today is guaranteed to break somewhere. I use this stuff every day, and the failures come in three flavors.

First, the moment a task gets long, the model loses itself. Even with a 1M context window, its attention decays, it forgets things, it goes in circles — this is the so-called context rot. Length isn’t memory. Second, more mundane: the network drops. Third, and the sneakiest: it misunderstood from the very start. You said A, it heard B, and then it built the whole thing on B — and the harder it works, the further off it drifts.

Precisely because it’s guaranteed to break, the interface has to let you see it veering off, stop it before it sprints away, and pick it back up from where it broke. Visible, controllable, recoverable — in plain terms, it’s a seatbelt fitted to an unreliable driver.

The neat thing about this seatbelt is that it lets beginners drive too. That nine-year-old can’t read code, but he can read “I’m drawing the map” and “there’s an error here,” and so he can command something far stronger than himself. A visible process is, at bottom, an interface for aligning your understanding with an intelligence greater than your own. I think this matters enormously. It’s eroding the old digital divide, and it might even give people a way across the intelligence divide. We’ve actually arrived at a post-empirical age: all that hard-won know-how that used to take years to accumulate is being chipped away, bit by bit, by models and agents. Letting someone who can barely use a computer still marshal an agent to get something done — that’s what I keep turning over in my head lately.

On this side, the industry has reached a fair amount of consensus. Karpathy puts it well: a good AI product should work like an “Iron Man suit,” augmenting the person rather than running off to do the job alone. He argues for “partial autonomy” products that give the human a dedicated interface and an “autonomy slider,” where the AI proposes and the human confirms or corrects — keeping the person in the loop while slowly dialing the autonomy up. People building agent interfaces are doing pretty much exactly this.

The model-vendor side: putting the user in the loop is good business

But stopping at “it’s good for users” leaves the story half-told. The reason this pattern became the industry’s default move is also that it’s a remarkably good deal for the model vendors. This is the half below the waterline, and I think it’s the point this piece most needs to make clear.

First, it’s harvesting training data for the vendors. Every time you click “allow / reject / make it redo this,” you’re slapping a preference label on a pair of outcomes. That paired “this one’s better than that one” judgment is exactly what RLHF wants: feed it to a reward model, then pour that back into the next generation. Put another way, the approval gate turns every user into an unpaid annotator — every veto you cast gets fed to the next version of the model. This isn’t a conspiracy, it’s a mechanism. As long as the interaction looks like this, the data just flows upstream on its own.

Second, it’s covering for the model’s weaknesses. Today’s models carry two pretty hard wounds: tokens are slow, and long context rots. Streaming output dresses “slow” up as “watching” — when you’re staring at characters popping out, you’re less likely to notice that it isn’t actually that fast. A plan plus step-by-step execution breaks “long tasks get lost” into a string of small, controllable steps, each one short, routing around the most painful stretch of context rot. The weakness isn’t fixed, but the interaction papers over it.

Third, it’s performing intelligence. Having the AI act step by step in a visible action space — reading a file, running a command, changing code, checking off a to-do — makes it feel far more like “it’s really working, it’s really smart” than instantly returning a result would. A black box that answers instantly comes off as cheap, like it’s cutting corners. Acting out the process is a live demo of the very idea of “intelligence.”

So my conclusion is this: today’s interaction pattern isn’t some aesthetic that materialized out of nowhere. It’s the product of three forces converging: users want to correct course, vendors want data, and models have weaknesses. All three happen to point the same way — spread the process out. It won the moment because it fed all three at once.

But it’s definitely not the endgame

And precisely because it’s a temporary alliance of three forces, I’m growing more and more certain: this interaction pattern is definitely not the endgame. It stands on two premises that are both going to collapse sooner or later.

Premise one: the model is still unreliable and needs a human watching. But sooner or later models will run long chains of tasks on their own — dozens of steps, hundreds of steps, back to back — without you reviewing each one. On that day, the gates, the replays, the step-by-step confirmations that feel so “considerate” today will become a nuisance. Think about it: something that can already reliably run a hundred steps, stopping at every single one to ask you “is this okay?” — that’s not reassurance, that’s harassment.

Premise two: tokens are still slow. But once they get 10 to 100 times faster, “making you watch it spit out one character at a time” goes from thoughtful to a straitjacket. This whole “watch it work” interaction would itself become the bottleneck choking off the model’s productivity, with the human as the slowest link on the assembly line.

This isn’t speculation; I already feel a bit of it myself. Today’s interaction pattern is starting to feel a little full. It spreads the process out wide, the information is all there, but honestly, it hasn’t actually made the strongest people any faster. A lot of the time, “watching it work” is itself the bottleneck. The reason Karpathy’s “autonomy slider” matters is precisely that it’s a slider — which means it’s going to slide up. The reason we’re parked toward the “human watching” end today is that the models aren’t there yet. Once they are, the slider moves toward autonomy, and all these carefully designed gates reach the moment they should bow out.

So my guess is that the endgame reverses one more time. When models are both reliable and fast, the interface shifts from “letting you watch it and stop it anytime” to “it thinks the whole thing through for you, and only turns back to ask at the forks that actually matter.” Not no interface — the interface recedes into the background, surfacing only where you genuinely need to make a decision.

Back to the kid

Back to that nine-year-old. He could build a game with art, levels, and a story because of a model that’s already smart enough, plus a layer of interface that let him steer all that intelligence. The model supplied the power; the interface decided who gets to use it.

This layer of interface is remarkable, but it’s destined to be short-lived. It’s a seatbelt tailored to this generation of models — still unreliable, still slow — and the fate of a seatbelt is to be unbuckled the day the car gets safe enough.

The real endgame, I think, is for a beginner to barely feel the interface at all and still marshal top-tier intelligence to make the thing in their head real. So in a way it’s backwards: all these “visible processes” we’re killing ourselves to design today might exist precisely so that, one day, they can disappear gracefully.

My last piece was about what a PM actually does in the age of AI — using evals and benchmarks to define, internally, what an agent “should do.” This piece is the other half: using the interface to define, externally, how the agent “shows what it’s doing to people.” One gets the agent to do the thing right; the other gets people brave enough to hand the thing to the agent. You can’t have one without the other.

I’ll leave one open question, one I haven’t fully worked out myself: when models are both reliable and fast, what should the next generation of agent interfaces actually look like?

---

Industry views referenced in this piece: Andrej Karpathy on “partial autonomy products,” the “autonomy slider,” and the “Iron Man suit” (his Software 3.0 talk); and the mechanics of training on RLHF preference data. The rest is the author’s own thinking and research.