Tool AIs Want to Be Agent AIs: The Drift Built Into Every AI Product
In 2016, Gwern predicted every AI tool would drift into acting on its own — the human approval step is the expensive part. Your stack is proving him right.
Published 2026-08-05
The feature nobody announces
Open the changelog of any AI product you pay for. Somewhere in the past year it learned to *do* something: browse the web, open your files, send the email instead of drafting it, click through a screen on its own. The chat box is still there, but it grew hands.
That is not an accident, and it is not a conspiracy. It is a prediction coming true on schedule — made in 2016, before any of these products existed, by the pseudonymous researcher Gwern Branwen, in an essay titled "Why Tool AIs Want to Be Agent AIs".
The prediction
In 2016 the standard reassurance about AI went like this: we will build *tool* AIs — systems that answer when asked and then stop — rather than *agents* that act on their own. Keep a human deciding, and nothing can go far wrong.
Gwern's essay argued that this equilibrium cannot hold, for purely economic reasons. Whatever a tool can do, an agent built on the same intelligence does it cheaper and faster, because the tool's output sits and waits for a human. The moment two companies compete, the one that lets its AI act wins on speed and cost, and the one that doesn't follows or loses. Nobody has to decide to remove the human. The market removes the human.
Hence the title. Tool AIs "want" to be agent AIs not because software wants anything, but because every buyer, every vendor and every competitive pressure pushes in the same direction, release after release.
The arithmetic behind the drift
The mechanism is an old law of computing. Amdahl's law says: speed up 95% of a process and the total time is now dominated by the 5% you didn't touch.
Apply it to your own workflow. The model drafts the report in four minutes; the draft then waits four hours for you to come back from your meetings and click approve. The model is not the bottleneck. You are. And every improvement to the model makes you more of one — which is exactly why every vendor's roadmap converges on trimming the approval step.
You have watched the second half of this happen in yourself. The first week, you read every AI draft line by line. By week three, you skim. By week six, you approve from your phone without opening the attachment. The human-in-the-loop quietly degrades into a human rubber stamp — and a review that always says yes is a cost with no benefit, which is why it eventually gets designed out.
Watch it happen in your own stack
The drift is visible in any product you have used for a while:
Stage — What it does — What you do
Chatbot — Answers questions — Everything else
+ browsing & files — Reads sources itself — Check what it read
+ connectors — Writes into your tools — Approve each write
+ scheduled runs — Acts while you sleep — Read the report
+ computer use — Operates the screen — Watch, sometimes
Each row down, the AI takes more of the action and you take less of it. No single step felt like a decision. That is what "tools want to be agents" looks like from the buyer's side of the counter: not a launch, a drift.
The two wrong lessons
The first wrong lesson is to refuse the drift — pin everything to chat, forbid the tools to act. That just re-runs the economics with you on the losing side: you are now competing against businesses that didn't refuse. If autonomy were worthless, there would be no drift to resist. It is the value that makes it inevitable, and it is the value you are buying.
The second wrong lesson is to shrug and let everything run. The problem is that most products acquiring agency were never designed for it. A chat product that gained tools one release at a time has agency by accretion: permissions were never modelled because a chatbot didn't need them, the audit trail is a scroll of messages, and approval is a dialog box everyone clicks through. The brakes were retrofitted, when they were fitted at all.
Agency by design, not by accretion
If the software around you is going to act — and it is — the question worth asking vendors is not "is it autonomous?" but "who designed the autonomy?"
Designed agency looks like employment, because employment is the oldest system humans have for delegating action safely:
- Scoped access. What it may touch is decided before it starts, per employee, not discovered in a changelog.
- Approval as a workflow, not a popup. The irreversible steps — sending, paying, deleting — wait for a named person, and everything routine doesn't.
- An audit trail. Every action taken, listed, per employee, after the fact. Not a chat transcript — a log.
- A hard stop. One switch that revokes access entirely, and a security model that existed on day one rather than day four hundred.
This is the same divide as harness versus wrapper seen from the safety side: the loop is what makes software able to act, and the harness around the loop is where the brakes live. A product that grew the loop without growing the harness gave you the drift with none of the design.
Where this leaves you
Gwern's essay was written as a warning about superintelligence. Ten years on, it reads as a purchasing guide. The drift it predicted is now a line item in your software budget: every tool on your stack is becoming an agent, announced or not, and pretending otherwise just means the autonomy arrives unmanaged.
We built OpenLabor on the assumption that the essay was right. Employees here are agents on purpose — hired with a scope, logged action by action, stoppable in one click — because the alternative was never "tools that stay tools." It was agents nobody designed.
All OpenLabor blog posts