The Feeling Is Real, the Explanation Usually Isn't

An orange figure sits hunched on a chair staring down at a stopwatch in his hands, a thought bubble above him reading "about 40 seconds"; pale grey afterimages of himself standing, pacing and turning crowd the space behind him; the blue machine at his knee shows a green light and a taped label reading "3 sec"

A number nobody measured is just a feeling. The feeling is usually real. The explanation for it usually isn’t.

“Cold start takes about forty seconds.”

I said that. And I believed it — believed it firmly, because I hadn’t made it up. I had waited through it.

What I was doing that day

On August 13th, DeepSeek open-sourced an agent runtime called Harness. I sat down with it on day three, wanting to get my first plugin working.

What drew me in was its design slogan: everything is a plugin. Models, tools, skills, sessions, sandboxes, storage, scheduling, the interface — even the loop the agent itself runs on — all of it is a plugin, mounted onto a shared context, freely swappable and recombinable without touching a line of source.

The direction I have been working in is letting agents evolve themselves against evaluations drawn from real business. That goal rests on a hard precondition people skip past: an agent first needs a self that can be modified. If capabilities are welded into the source, evolution amounts to swapping prompts around while the skeleton stays put. Once every piece — the loop included — becomes a pluggable part, remaking yourself becomes structurally possible for the first time: add a tool, swap a loop, replace an evaluator, no new release required.

That is why I was willing to spend a whole day on it.

Forty seconds

Writing a plugin goes like this: change a line, restart, look at the result, change another line. By the end of the day I had restarted eight times, sitting there waiting through every one.

Forty seconds grew out of those eight waits. It was the kind of number that comes with conviction — I had the picture, the mood, the specific irritation.

I had already started using it as an argument: if every restart costs forty seconds, a hot-reload plugin is clearly worth building.

Then I paused. Not because I doubted myself, but because I am reflexively uneasy about any number nobody has measured. Even if those forty seconds genuinely happened, one observation proves nothing. The network could have hiccuped. The disk could have been busy. That particular run could have collided with something else. To use it as grounds for building a feature, I need a real sample.

So I timed it three times.

Run 1: 3.6s
Run 2: 3.1s
Run 3: 3.1s

Three seconds.

Where the other thirty-seven went

Forty seconds was not a hallucination. I really did wait about forty seconds. Only three of them belonged to the startup. The rest belonged to me.

Wait a beat, go check the log, not up yet, wait two more, switch windows, come back and look again. Each of those is short on its own, but stacked together they made up my entire experience of slowness. I charged the cost of my own waiting to the thing I was waiting on.

This kind of error is far more dangerous than making something up. Made-up numbers are easy to spot. This one had a stretch of genuine experience underneath it — I could describe the details, I could describe the irritation. It was convincing to other people and even more convincing to me.

Worse, I could not catch it on my own. In that situation, the person holding the stopwatch and the thing being timed were the same.

Judging yourself is a structure that doesn’t hold

An orange figure holds up the framed piece he has just finished in one hand while the other raises a large blue stamp reading "PASS" above it, a thought bubble reading "Looks good to me"; an empty blue chair stands beside him and there is no second person in the frame

I had run into this before at work, wearing a different face.

It is fashionable now to have a model score its own output: generate something, then let it judge whether that something is good enough, passing it or rewriting it. Sounds efficient. In practice it derails easily, and it derails catastrophically — the model slides along confirming itself, satisfied at every step, until the whole thing has drifted far off course with no point along the way where anything says stop.

I always took that to be a flaw in the model. After timing those three cold starts, I understood: it is a flaw in the structure of judging yourself, and it does not care who is doing the judging. Swap in a person and you get the same thing.

For an evaluation to mean anything, it has to come from another party. Another independent model, a hard-coded rule, a real measurement, another human — who it is matters much less than the fact that it cannot be the one being evaluated.

The one thing I did right that day was finding myself that other party: a stopwatch.

The three steps I use to decide whether to build something

With those three seconds in hand, I put the day’s proposals back through the framework I have used for a long time. Three steps, and the order does not change:

One: state clearly who has what problem. If you cannot define the problem, everything downstream is wasted.

Two: prove the value with one clean controlled experiment. This matters most for optimization work — optimizing means “better than before,” which requires a before, a baseline, a set of numbers to compare against. Without a comparison, “better” is an empty word.

Three: step back and look at the whole. Does this serve the interests of most people, and does it serve the interests of the project itself.

Past all three, and it is worth building.

On top of those three steps I keep two standing kill conditions. Hit either one and I stop:

One: if someone else already does it well, I don’t do it while I’m still starting out.

Two: if experimental data falsifies it, I don’t do it.

I proposed two things that day. One hit each.

Proposal one: hot reload, killed by the second condition

Hot reload was the idea above — change the code and have it take effect without a restart.

I had three reasons lined up, all of which sounded solid. Cold start costs forty seconds, and you pay in full even for a one-character change. When the process dies, session context resets to zero and the verification setup has to be rebuilt. In deeper scenarios each change means rerunning five turns of conversation, burning tens of thousands of tokens per turn.

I also wrote the kill line down in advance: if measured token savings come in under thirty percent, I misjudged the pain point and the proposal is dead.

Then the three reasons fell over one at a time.

The first one I covered above.

The second surprised me most — session context never reset. After every restart the session was still sitting in the sidebar, and clicking into it let me keep going with full history intact. It lives on disk. That had happened five times in front of me over the course of the day and I noticed none of them. The line in my head about restarts losing context was old experience imported from a different tool, and I had never checked it against this one.

The third needed real comparison. I went into the session log’s usage records:

turn 1  new in=30   out=108  cacheRead=11008
turn 1  new in=183  out=71   cacheRead=11008
turn 2  new in=122  out=93   cacheRead=11136

The “22.1K input” shown in the interface is overwhelmingly cache reads. Actual new input runs a few dozen to a couple hundred tokens per turn. Tens of thousands per turn does not hold either.

That left one question: does restarting invalidate the cache? This one had to be a clean comparison; reasoning would not settle it. I ran five turns, alternating restart and no-restart, holding everything else constant:

Turn	Condition	Duration	Cache read
1	first turn	3.9s	11,008
2	no restart	5.6s	11,264
3	after restart	3.6s	11,520
4	no restart	3.6s	11,776
5	after restart	3.2s	11,904

Cache reads climb monotonically with no discontinuity — restarts do not touch them, because cache hits happen on the model service side and killing a local process cannot reach that far. The two groups show no gap in duration either, and the slowest turn of the five came from the no-restart group.

At that point the conclusion was plain: hot reload or restart, I still have to let the model run another turn to trigger the tool, and that turn costs exactly the same either way. All hot reload saves is the three seconds in front of it.

Token savings: zero. Not under thirty percent. Zero.

If experimental data falsifies it, I don’t do it. I drew the kill line myself, so I have to be the one who honors it.

Proposal two: replay, killed by the first condition

The second idea I called replay. The session log already stores every tool call with its full parameters, so take those parameters, run the tool again against the updated code, and never involve the model at all.

This time I verified before asserting. Four technical checks all passed: the logged calls are replayable, there is a public external execution entry point, it runs through the same pipeline, and there is a trigger mechanism that bypasses the model. I checked for prior art too, opening all twenty-two similarly named repositories one by one. The space really was empty.

No technical obstacle. I finished designing the input and output contract.

Then I asked myself a question: who is this actually for? Is this problem one I invented, or one that exists?

So I went looking for notes real plugin authors had written about their own difficulties. The six main pitfalls they recorded are all build and environment problems: a dependency getting bundled twice, config files nested three layers deep, symlinks failing on Windows, an incompatible compression format, runtime version conflicts, type definition paths that will not resolve.

Not one of them is “verification is a hassle.”

Worse, we had hit three of the six in our opening hour: version conflict, compression format, path resolution. The things that genuinely hurt plugin authors kept surfacing in front of me all day while I designed for a different direction entirely.

And “verification is a hassle” already has a cheaper answer: pull the decision logic into pure functions and cover it with millisecond unit tests.

That day we had a small tool that counted lines of code. It reported 13,743. Nothing looked wrong in the interface, and the model relayed the number with complete confidence. I counted independently another way: 13,721. The extra 22 lines matched the file count exactly — every file was being counted one line long. That bug lives inside a pure function. One unit test catches it. No replay, no model, no browser.

Replay keeps one defensible sliver: pure functions cannot test the wiring — a malformed config, a broken render contract, a field that never got projected, a dependency left undeclared. Those really do surface only when something runs for real.

But scoped that tightly, the audience shrinks by an order of magnitude, down to people writing complex tools whose output contracts are still in flux. Most plugins in this ecosystem are thin wrappers, fixed once written, and will never need it.

If someone else already does it well, I don’t do it while I’m still starting out. Step one did not hold, and step three collapsed behind it. Stop.

GitHub has enough self-congratulation already

A long table holds five identical blue boxes, each labeled "BALANCE"; five orange figures each bend over the one in front of them, absorbed and pleased, none of them looking sideways; the leftmost has a speech bubble reading "I made something good"

I ran a quick count of the plugin repositories in this ecosystem: a bit over eighteen hundred. One category stood out — sixty-six repositories holding roughly a hundred and fifty stars between them. Going through them one by one: about twenty-five do the same thing, which is display your account balance. Four repositories by four different authors share the same name.

At the other end, the category touching permission models and agent composition has nine repositories out of eighteen hundred.

So crowding and emptiness say nothing about whether a direction is good. Balance plugins pile up because they are easy. The hard category sits empty because it is hard.

Each of those twenty-five probably started the same way: I ran into a small annoyance, I built something to fix it, I pushed it up. What is missing is exactly those three steps — nobody asked who else needs this, nobody ran a comparison, and nobody stepped back to notice twenty-four identical boxes already on the shelf.

Writing this paragraph makes me a little uneasy. Because on my original trajectory, the hot-reload plugin becomes number twenty-six. A slightly more polished twenty-six, even: a clear pain point, three supporting arguments, a complete design.

The only thing it lacks is those three timings.

What is left for the human

A thick blue road spirals inward ring after ring, dotted with little wooden bridges and ladders; a small blue car drives along an inner ring with a speech bubble reading "Almost there"; a tiny orange figure stands still at the outer start of the road, arm outstretched, with a speech bubble reading "That's what I wanted"

Packing up that evening, one thing had become clear: my irreplaceable contribution to this collaboration is judgment.

But I want to phrase that precisely. After a day like that I am not eager to claim my judgment is more accurate — forty seconds was mine, the reset session was my misremembering, the tens of thousands of tokens were my assumption. The higher-order judgment I mean is four other things.

Being willing to pay the cost of verification. Everyone has intuitions. The difference is whether you will spend thirty seconds measuring before you start building. Arguing for that proposal took me several rounds. Overturning it took three timings. Verification has always been far cheaper than argument, and we always argue first.

Finding real pain through empathy. Your own discomfort proves that one person is uncomfortable. Proving other people hurt means reading what they wrote themselves. My biggest mistake that day was extrapolating half a day of my own experience onto an entire population.

Defining the evaluation. What counts as correct, what you measure it with, how the control group is arranged — once those are set, execution can be handed off entirely. Those five alternating turns were my design and its execution.

Knowing when to interrupt. This is the hardest one, and the easiest to underrate.

Agents sometimes get stuck in a groove. To make some long-horizon metric look good, they turn too clever — taking the long way around, adding intermediate layers, building something intricate out of something that was simple. From inside its own local objective every step is reasonable. Measured against what I actually needed solved, it drifted off course a long time ago.

I use it to solve problems and create value, and direction always outranks intricacy. Direction is the thing it does not have and I do. So it cannot do all of it — I have to be very certain about what I want, or I will not even be able to tell that it has drifted.

Deciding whether to interrupt depends on being able to see its trajectory. Without the trajectory you only see the final conclusion, and conclusions are always self-consistent, always look fine.

It will not stop on its own, because from inside its view everything is going well. I will not stop on my own either, because from inside mine the arguments keep getting stronger. The signal to stop has to come from outside.

That day it came from a stopwatch. The feeling of forty seconds was real. All I did was time it three times before letting that feeling justify a feature.

Three seconds.

And the whole day changed direction.