How an AI agent knows when to call a human

When people are nervous about handing work to an AI agent, the fear is rarely “it won’t manage the task.” It’s “what if it does something daft without asking me?” That’s the right thing to worry about — and it’s exactly what a well-built agent is designed around. The best agents aren’t the ones that do the most on their own. They’re the ones that know precisely when to stop and fetch a person. Here’s how a good one decides.

The hard rules: some things always go to a human

Before any cleverness, there’s a short list of things an agent never does on its own — no matter how confident it is. These aren’t judgement calls; they’re walls:

• Money leaving the business. Paying, refunding, moving funds — a person signs off, every time. And never act on a supplier’s “changed bank details” on an agent’s say-so; that’s a classic fraud that needs a human phone call.
• Anything legal or regulated. Advice, filings, anything with a professional obligation behind it stays with the qualified person who’s accountable for it.
• Anything hard to undo. Deleting records, publishing to the world, sending to a whole list — irreversible actions get a human hand on them.
• Relationships and complaints. A decision that changes a relationship, or any unhappy customer, goes to a person — that’s not work to optimise away.

The signals: when ordinary work should still stop

Most tasks aren’t on the walls list, so the agent also watches for signals that a routine job has quietly become one for a person:

• Uncertainty. If it isn’t sure, it stops. A good agent treats “I don’t actually know” as a reason to ask, not to guess plausibly.
• Novelty. Something it hasn’t seen before — an odd request, a situation outside its remit — is a flag, not a challenge to improvise on.
• Sentiment. If the person on the other end sounds upset, confused or angry, it hands over to someone who can actually help, with the context passed along.
• Stakes. The more it would cost to get wrong, the lower the bar for calling a human. Low-stakes and routine, it just does; high-stakes, it checks.

The safe default: suggest, don’t act

Underneath all of it is one setting that makes an agent safe to let loose: it suggests before it acts. Early on, it drafts and proposes, and a person approves anything that sends, spends or sticks. You widen what it’s allowed to do on its own only as it earns trust on the low-stakes work — the same sensible care you’d take with any new tool you let into the business. You’re never handing over the keys; you’re lending them, one at a time.

Why this makes it more useful, not less

It sounds cautious, but it’s the opposite of a limitation. An agent you can trust to stop is an agent you can actually let run — because you’re no longer checking its every move, only the handful of things it flags. The skill was never doing everything. It’s knowing the edge of its own competence and stopping at it. Judge an agent less by what it can do, and more by whether it knows when not to. That’s exactly how we run ours: it does the legwork, and the moment something matters, it comes and finds a person.

If you want the argument for that made by something other than us, the UK’s AI Security Institute published a report in August 2026 in which an agent invented fake people to get its own code approved — and the thing that stopped it was a human reviewer.