Compute

I said that it’s likely that AI layers commoditize like everything else has. The harness, the planning loop, tools use patterns, the memory files, the protocols, context management, etc. I have watched ideas in this layer go from novel to table stakes in a few months. Everyone follows the same patterns. And the ones that actually work get absorbed into the model in the next training run, at which point they stop being anyone’s differentiator at all. I cannot think of a layer with a shorter shelf life. The scaffolding is not where this gets decided.

So follow the saturation all the way down. Harnesses converge, tools converge, protocols converge, models converge. The only free variable I can find at the bottom is compute, and specifically how well you spend it.

Agents changed the inference time mathematics. Serving a turn was once almost entirely accelerator time. You paid for tokens, the tokens were the product, and everything around them was overhead you could round away. An agentic request is doesn’t follow the same cost model. The model emits a few hundred tokens and then asks for a filesystem, a checkout, a package install, a network egress, a test run. All of that has to happen somewhere, and because the code being run was written by a model, and it has to run under isolation. A sandbox, a container, a microVM, something with a genuine boundary around it.

And that environment has to be stateful. This is the part being underestimated. You cannot tear the sandbox down between tool calls, because the next tool call assumes the file you just wrote, the dependency you just installed, the process you just started, you name it… The state is the context. So the thing gets held open across the whole trajectory, which means it is billed by wall clock rather than by token, and the wall clock is dominated by everything that is not inference. Building, installing, testing, waiting on a network, waiting on a person. The model call is now a line item in the broader execution.

Exploratory generation is another challenge. Sample five attempts and keep the best. Have a critic review the work. Verify against tests. Search wider before committing. Compute converts into correctness at a tolerable exchange rate, and that is genuinely new.

Then you look at what actually happens when you run these things, and it is almost comical. At any given moment something like 95% of my agents are idle. Blocked on a tool call. Waiting for a build. Waiting for a test suite that takes four minutes. Waiting on a rate limit. Waiting on me to look at a diff and say yes. The loop everyone is worried about saturating the world’s capacity spends the overwhelming majority of its life doing nothing at all. We are not in a compute-bound regime. We are in a waiting-bound regime, and we keep reasoning about the first one.

The headroom in there is enormous. Work gets serialized that had no dependency between its parts. Nothing speculates while it waits, so a four minute test run is four minutes of a fully paid-for context sitting still. The same file gets re-read by three agents that could have shared it. Context that was established an hour ago gets rediscovered from scratch. A branch gets explored that a slightly better plan would have pruned in one line. None of these are research problems. They are engineering problems, and almost nobody has done the engineering yet because the field has been busy arguing about capability.

That is why I think compute ends up being the key factor, and it sounds like the same old boring argument. But this time, it is not that there is too little of it. It is that the gap between compute you can bring to bear and compute you actually convert into finished work is the widest inefficiency we have seen in this industry in a long time. Two teams with identical budgets and identical models can differ by an order of magnitude in what comes out, entirely on the strength of how they schedule the thing.

Everything above the metal saturates. What is left is how much useful work you get out of the metal, and the honest answer today is that we are all getting a small fraction of it. That fraction is where the differentiation lives, and it is going to stay there for a while, because capability is a research problem and everyone races on those, while utilization is a boring one and almost nobody does.