The model that decides and never clicks
A model with no browser, no tools and no memory between calls is the most useful kind, provided you hand it a closed set.
Most browser automation puts the model in the driver's seat. It gets a page, a set of tools and a loop, and it decides what to do next with no idea whether the next step can actually be performed.
I run it the other way round, and it holds up better in production. The model never touches the page. It receives a description of the current state and returns a judgement. Separate code performs the action.
What a decision model is
The distinction matters, because the word agent has been stretched to cover both.
A decision model has no browser, no tools and no memory between calls. You send it a state, which is text. You send it questions. Each question has a fixed answer type.
- A probability, for a yes or no judgement.
- A choice, from a list you supply.
- A score between 0 and 1, for a graded judgement.
That is the whole interface. It cannot navigate, click, type or retry. Ask it to do those things and you have misunderstood it. Ask it to judge, and it is fast, cheap and consistent.
The surrounding code, which I call the harness, does everything else. It reads the page, builds the state string, sends the questions, parses the answers, performs the action, and then loops back to read the page again. Observe, decide, act, observe.
Why this beats an open-ended loop for real work
Three reasons, and none of them is about capability.
- Every decision is loggable. The model's answer is one of a closed set, so the log records choice 27 of 33 with a stated confidence, not a paragraph of reasoning that a person has to interpret later.
- The action layer is testable on its own. Whether a click landed is a question about the harness. You can test that with no model in the loop at all, which means a failure tells you which half broke.
- The blast radius is small. A model that can only choose from a list cannot invent a step. The worst outcome is a wrong choice from a known set, which is recoverable and usually obvious.
The failure mode, which is the useful part
A decision model is exactly as good as the options you hand it.
Early on I asked one to find the reply control on a webmail page and click it. The state described the page accurately. The model returned a reasonable answer, and the answer was useless, because the control had no readable label and had never been in the option set. Nothing in the output said that. It answered the question I asked, and the question was wrong.
The fix is to stop asking open questions and start enumerating. I now walk the page, tag every clickable element, and hand the model the numbered list as the closed set. On the page that produced the useless answer there were 33 candidates, one of which was the reply control. Given the list, the model picked it with a stated confidence of 1.00.
The lesson generalises. When a decision model gives you a plausible answer you cannot act on, the fault is almost always a missing option, and not a weak model.
Turning a judgement into a rubric
The second use is the one I did not expect to matter as much as it does.
Any judgement a person makes repeatedly can usually be converted into a small set of closed questions, and then it stops being a judgement and becomes a measurement. I wanted to know whether the AI answer a search engine gives for a name states that a spent restriction has ended. Read as prose, that is a matter of opinion. Broken into questions, it is a rubric.
- Is an AI-generated answer present? A probability.
- Does it state the end date? One of none, implied, explicit.
- What date does it give for the associated fine? One of 2017, 2018, not stated, other.
Run that on every reading and you get a series rather than an impression. The same answer scored twice comes out the same way, and a reading from three weeks ago is comparable to one from today.
Where the person stays
I keep a human at three points, and I would not move them.
Logins are the person's, because credential handling in automation is a risk with no upside. Irreversible actions are the person's, so the model proposes and a person sends. And the decision about whether the whole task should happen is the person's, because that is a question about intent, and intent is not in the state string.
A decision model does not remove the person from the loop. It removes the part of the loop that was always mechanical, which is choosing between options that were already known.
The honest limitation
The model has no memory between calls. Everything it needs must be inside the state you send, and that includes the reason the task exists. Send it a thin state and you get a confident answer about the wrong thing.
It also cannot tell you that your option set is incomplete. That failure is silent, and it looks like a model problem when it is a design problem.
I build these systems for a living, so I have an interest in the approach. The reason I settled on it is narrow and practical. It is the only arrangement where I can read the logs afterwards and say exactly what was decided, by what, from which options, and whether it worked.
AutoCoreAgent builds custom AI systems for businesses: voice agents, outreach automation, workflow automation and the operational parts most projects skip.
AutoCoreAgent →