Agentic AI Governance: From Pilot to Production

An agent that works in a demo and an agent that runs in production are two different objects. What sits between them is agentic AI governance: an approved access scope, an action log, an escalation threshold, a cost ceiling. Among large enterprises, 40% of respondents report scaling AI agents, up from 27% a year earlier. Six in ten are still stuck at the pilot. This article sets out what has to exist before go-live, and why the thing blocking it is almost never technical.

What is agentic AI governance?

Agentic AI governance defines what an agent is allowed to do, how far it goes without human sign-off, and what happens when it gets something wrong. It is decided before production, not after the first incident.

What separates it from AI governance generally is one word: action. A generative model produces output that someone reviews before using it. An agent executes. France’s data protection regulator, the CNIL, writing with the Conseil de l’IA et du Numérique in July 2026, describes a technology now able to act on its environment in place of its user. That delegation, not model capability, is what changes the thing you have to control.

Governing a model means bounding what it produces. Governing an AI agent means bounding what it can do inside your systems.

Why do agents that work in a pilot never reach production?

Because a pilot asks none of the questions that block production: who approved the data access, what the agent does when a step fails, what one run costs. A pilot proves feasibility. Production requires operability.

Gartner forecasts that over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls. None of the three is a model problem.

The pattern repeats. A team builds an agent that works, demos it, gets approval in principle. Then risk sees the scope of access being requested, nobody can say what the agent did last Tuesday, and cost per run was never calculated. The project doesn’t stop because it failed. It stops because nobody can sign it off.

What has to be in place before go-live?

Five guardrails, all decided before launch: access scope, action log, escalation threshold, cost ceiling, exit condition. Each answers a question someone will ask anyway, and the only variable is whether you answer it early or under pressure.

Access scope lists the data and systems the agent can read, and the ones it can write to. Without it, the security review discovers the real extent of the access being requested three weeks before launch, at a point where the only fast decision available is to postpone.

The action log records what the agent did, when, and with which data. Without it, no incident can be reconstructed afterwards: you know one record is wrong, but not which run produced it, nor how many others took the same path.

The escalation threshold sets the failure count, or the action type, that hands control back to a human. Without it, the agent retries instead of stopping, and the default behaviour of a failing loop is to spend budget repeating the same mistake.

The cost ceiling sets maximum spend per run and per month. Without it, consumption is discovered on the invoice, a month after the point at which it could have been corrected.

The exit condition defines what ends a run: a result obtained, an error, a maximum number of steps. Without it there is no guarantee a run terminates at all, and it is the one guardrail of the five whose absence is a design fault rather than a governance gap.

Which guardrail actually delays go-live?

Access scope and the escalation threshold. They are the only two of the five whose decision sits outside the project team, and they are the ones production launches stall on.

The other three are set by the team that built the agent: the action log, the cost ceiling and the exit condition are engineering decisions, taken by people already in the room. Access scope is not. It belongs to an IT or security owner who never saw the pilot. Nor is the escalation threshold, which belongs to a business team that will have to absorb the handovers. So the realistic timeline for a go-live is not the development timeline. It is the timeline of those two approvals.

And the first of the two isn’t a technical problem. It’s triage. At Club Med, the programme surfaced 102 candidate agents with 50 ambassadors trained over two days. Eighteen were kept as quick wins ready to deploy, 57 documented as structural projects. The work that mattered wasn’t finding 102 ideas. It was setting 84 of them aside. A portfolio of 102 agents cannot be governed, because no security function will approve 102 access scopes; a portfolio of 18 can.

How do you keep control without removing the agent’s autonomy?

By layering several narrow controls rather than one broad one, and by defining in advance what triggers a human takeover. An agent with no escalation threshold isn’t autonomous. It’s unsupervised.

OpenAI frames these protections as a stack: while a single one is unlikely to provide sufficient protection, using multiple, specialized guardrails together creates more resilient agents. The guide names relevance and safety classifiers, a PII filter, moderation, tool safeguards, rules-based protections and output validation.

On human intervention, two triggers are named outright. The first is a failure threshold: set limits on agent retries or actions, then escalate once they are exceeded. The second is high-stakes actions. Those that are sensitive, irreversible, or have high stakes should trigger human oversight until confidence in the agent’s reliability grows. Both are one meeting’s work to define. Both are missing from most pilots.

How do you know the governance is working?

You measure it like anything else: escalation rate, average cost per run, time to approve an access request, share of actions logged. Governance that produces no numbers is indistinguishable from no governance.

This is exactly where the leaders separate. McKinsey finds that high performers are much more likely than others to report working to mitigate unauthorized or unintended actions by their systems. The same survey, covering 1,719 respondents across 97 countries and published on 25 August 2026, shows nearly three-quarters of high performers have fundamentally redesigned workflows, up from 55% last year, while 37% of respondents attribute at least some EBIT impact to AI use.

An escalation threshold with no measurement of the escalation rate is an intention. With the measurement, it becomes a setting you can tune.

Where should you start?

With an inventory and a filter, not with tooling. List the agents already running and the ones in flight, rank them by how sensitive the access they need is, and write detailed rules only for the ones going to production.

That is what we frame during the audit and roadmap, over four to six weeks, before the first agent is built. The guardrails are then designed in rather than bolted on, which is the only sustainable way to get AI agents into production. And because an escalation threshold the business team doesn’t understand will never be respected, upskilling is part of the control system rather than a nice-to-have. That is the point of our training programmes.

Frequently asked questions about agentic AI governance

Does governance slow down getting to production?
It moves the delay rather than adding to it. Time spent approving an access scope up front is time you don’t spend arbitrating it under pressure three weeks before launch.

Do we need a dedicated AI committee?
Not at the start. A named owner per agent, a written access scope and a monthly review are enough while the portfolio stays under roughly twenty agents.

Is agentic AI governance different from AI governance in general?
Yes, on one decisive point: an agent acts inside your systems instead of producing output you review. Access, logging and escalation have no equivalent in governing a generative model.

Is an enterprise AI agent high-risk under the EU AI Act?
Rarely, but it depends on the use, not the technology. An agent involved in hiring, credit or access to an essential service is far more likely to be in scope than one reconciling data.

Want to know which of your agents can go to production, and what the others are missing? Book a 30-minute call and we’ll review your portfolio and its guardrails, with no preparation needed on your side.