Somewhere in your organisation there is almost certainly an AI policy. It was probably written in the last eighteen months, it references responsible use, human oversight, and data protection, and it was a genuinely worthwhile exercise. It is also, in most organisations we assess, doing quiet harm — because its existence is being mistaken for the capability it describes.

A policy is a statement of intent. A governance capability is the operational machinery that makes the intent true on a Tuesday afternoon when something goes wrong. Agents — software that acts, rather than software that waits to be used — expose the difference brutally, because they generate decisions at a rate no quarterly review was designed for.

Three questions that separate policy from capability

When we assess an organisation's governance readiness, three operational questions do most of the work:

"What did it do yesterday, and why?" If an agent has been acting on your organisation's behalf, someone must be able to reconstruct its decisions after the fact — not in principle, but with an actual log, owned by an actual person, reviewable in an actual tool. If the honest answer is "we'd have to ask the vendor", you have outsourced your accountability along with your infrastructure.

"Who does it escalate to, and do they know?" Every sensible policy says agents defer to humans on consequential decisions. The capability question is whether a named human receives those escalations, has the authority to resolve them, and has the time to do so. An escalation path that terminates in a shared inbox is a policy, not a capability.

"Can you turn it off without turning the business off?" The kill-switch question. If the agent has become load-bearing — woven into a process such that stopping it stops the work — then switching it off is no longer an operational decision, it is an incident. Ready organisations design agents to be stoppable: fallback procedures exist, and someone has rehearsed them.

Why this gap persists

The gap is not negligence. Policies are cheap to produce and satisfy real external pressures — boards, regulators, procurement questionnaires. Capabilities are expensive: they need tooling, role definitions, and rehearsal. And crucially, nothing tests the gap until an agent is live. Traditional software mostly waited for a human to click; the human was the governance. Agents remove that implicit control, and what remains is whatever explicit machinery you actually built.

This is also why we are wary of readiness assessments that audit documents. The documents are nearly always fine. The findings that matter come from asking people the three questions above and listening to the pause before the answer.

Where to start

Not with a bigger policy. Pick the single most consequential place an agent might act in your organisation and build the machinery for that one case: the decision log, the named escalation owner, the rehearsed stop procedure. A governance capability built around one real deployment teaches you more than a framework covering a hundred hypothetical ones — and unlike the framework, it scales by repetition rather than by rewriting.