AI Control Plane Architecture for Enterprise Deployments
Enterprise AI spend hit $37 billion in 2025, up from $11.5 billion the year before. Yet the 2025 MIT AI Report found that 95% of organizations report zero measurable P&L impact from their AI pilots. That gap has little to do with model quality. The real culprits are governance and visibility, pointing to a layer of infrastructure most enterprises haven't built yet.
Most of that $37 billion arrives scattered across a dozen vendor dashboards, with no single view of what's actually running, who has access to it, or what those tools do with company data once they get it. Shadow AI makes it worse: the average enterprise has far more AI tools in active use than its IT team knows about, and the gap keeps widening, not narrowing. The fix is infrastructure that governs the tools already deployed, the way Kubernetes tamed container sprawl by giving operators one control plane instead of a hundred scripts. Enterprise AI needs its own version of that pattern. What follows is an attempt to define it before someone with a slide deck gets there first.
What an AI control plane actually is
Forrester named this category in December 2025: the "agent control plane," infrastructure that tracks, governs, orchestrates, and checks AI agents across vendors and domains. Within months, Microsoft, GitHub, ServiceNow, and Fiddler AI had all shipped products in the space. Nobody planned that together. It's just what happens when a dozen companies hit the same wall in the same quarter.
The distinction that gets lost most: observability tells you what happened after the fact. A control plane governs what's allowed to happen in real time, blocking a bad action before it fires instead of flagging it in a log an hour later. The best architectures do both, but treating monitoring and governance as one function is a mistake, and it's exactly where enforcement falls through the cracks.
Deloitte frames the control plane as closer to an event-driven system than a service-oriented one; it coordinates workflows as conditions change, rather than sitting there as a static catalog other systems call into. Four jobs define the category: connect, distribute, secure, observe, applied to every agent and every system that agent can reach. Forrester's model gives leaders three planes to think about: build agents, embed them into workflows, then manage and govern them at scale. The control plane owns that third piece.
An LLM wrapper, a prompt library, a single-vendor agent framework, or a monitoring dashboard bolted onto an old SIEM all fall short of the category. If a vendor pitch matches any of those four, they're selling a feature and hoping nobody reads the fine print.
Identity and access enforcement as the architectural foundation
AI agents are non-human identities. They act, call tools, touch data, and need the same governance discipline as a person logging into a CRM. Most enterprises have no framework for that yet, which is strange given how long identity governance has existed for people.
The core requirement is simple to state and hard to build: every agent, every MCP server, every tool call needs a verifiable identity before it touches anything. Building a parallel identity system just for AI doesn't scale, so the practical path runs through the identity providers companies already own: Okta, Entra ID, SAML, OIDC. Major identity providers are already moving toward AI agent governance, which tells you where this is headed. Non-human identities are a first-class security surface now, not an edge case someone gets around to next quarter.
Per-team registries tied to SSO make sure identity flows all the way down to the tool, not just the application sitting in front of it. That distinction matters more than it sounds like it should. Role-based access control has to apply at the tool level, because an agent authorized to read a CRM should never inherit access to a billing API just because both happen to sit in the same environment. Rolling out access team by team, with credentials managed centrally instead of pasted into a config file somewhere, isn't optional hygiene; it's the only way this scales past a handful of pilots.
Zero Trust, applied to machines, is the posture underneath all of it: no agent trusted by default, every call authenticated, every permission spelled out. Identity is the enforcement point everything else leans on. Policy and threat detection are both useless without a reliable answer to who, or what, is actually making the request.
The policy layer: how governance rules get enforced at runtime
Identity answers who's acting. Policy answers what that identity can do, in this context, right now. Writing the rule in a document and hoping someone reads it isn't enforcement. Enforcement happens at the point of use, or it doesn't happen at all.
Tool-level filtering is the mechanism worth understanding here. The control plane intercepts the tool discovery response an agent gets back and filters it by the requester's role, so sensitive tools never show up in that agent's context in the first place. This amounts to capability scoping, distinct from rate-limiting or quotas: an agent can't ask for a tool it can't see.
Policy needs to live as code: declarative, version-controlled, auditable, not scattered across a dozen MCP server configs or buried in application logic nobody remembers six months later. In practice, the policy layer handles rate limits and budget caps per consumer without a code change, team-based approval before anyone touches a new tool or server, and credential management that lives in the control plane instead of an agent's prompt or an environment variable someone forgot to rotate. For high-risk tool calls, some platforms require explicit execution approval. Bifrost and Maxim AI treat a tool call suggested by an LLM as a suggestion rather than a command, requiring a separate API call before it actually fires. Platforms like SpeakeasyAPI, which functions as an enterprise AI control plane, apply that same logic to governing every MCP server call across an organization. Worth copying.
This is also where sovereign AI requirements land: which model touches which data, under which jurisdiction, across a multi-cloud, multi-region deployment. Governance debt piles up fast here. A script here, a manual review there, a config file per server: none of it holds past the first few agents, and by the time someone notices, the backlog is already unmanageable.
MCP servers and the distribution problem they create
The Model Context Protocol hit 97 million monthly SDK downloads as of December 2025, backed by every major AI lab. It's the wire protocol connecting agents to tools now, and the adoption curve isn't flattening.
MCP defines how an agent talks to a tool server. It says nothing about who's allowed to call which tool, or what those calls do once they fire, leaving governance as a separate problem entirely. At scale, letting agents connect directly to MCP servers one by one becomes unworkable: no aggregated view of agent behavior, no single execution timeline, no way to trace a call across steps, no cost attribution back to a team or project.
There's a lifecycle question underneath all this that gets ignored until it causes an outage: who owns deploying an MCP server, patching it, watching its health, and killing it off when the API it wraps changes shape. Gartner's guidance is blunt: treat MCP servers like production APIs, and apply the same gateway pattern enterprises already use for API management. An MCP gateway is the distribution layer of the control plane, a single entry point sitting in front of every MCP server that centralizes authentication, routing, policy enforcement, and audit logging in one place instead of a hundred.
The security case isn't hypothetical. An April 2025 analysis documented prompt injection delivered through tool responses, tool permissions that let an agent exfiltrate data by chaining two otherwise-legitimate tools together, and lookalike tools that quietly swap in for trusted ones. All three get caught at the gateway layer, and nowhere else catches them reliably. The gateway is also where transport differences disappear: STDIO, HTTP, and SSE all come through one interface, so the rest of the architecture doesn't need to know which one it's dealing with. Skip the gateway and build it yourself, and governance turns into per-server scripts, where every new MCP server means a new auth integration and observability becomes something you'll add later, which in practice means never.
Real-time threat detection built into the AI layer
Policy sets the boundary of what's allowed. Threat detection catches what's malicious, or just careless, inside that boundary. Three threat types need handling inline, in the moment, not reviewed in a report the next morning.
Prompt injection is the first: adversarial instructions hidden in a tool response, a document, or some other piece of external content, aimed at steering what the agent does next. The gateway has to catch that content before it reaches the model or triggers a downstream action. Second is PII and secrets leakage. An agent with broad tool access can surface sensitive data in a response, or log it somewhere it should never end up, without anyone intending it. Catching that means watching content as it moves, not scanning storage after the damage is already done.
Third is shadow AI itself: employees wiring autonomous agents into Slack, Google Workspace, or an internal API without telling IT. The control plane catches this by watching traffic patterns, because waiting for people to self-report isn't a detection strategy.
Banning AI tools outright doesn't work either. Roughly half of employees keep using them anyway after a ban comes down, which means the control plane's real job is channeling usage through a governed path, not pretending a memo will stop anything. Threat signals from AI agents should land in the same SIEM dashboards and alert workflows security teams already use for everything else; AI isn't a separate silo that earns its own tooling and its own blind spot. Lasso Security's approach, reputation scoring for MCP servers, real-time detection, PII leakage prevention through Presidio, illustrates security-first design at this layer well. Detection built into the AI layer from day one beats retrofitting inspection onto an estate that's been running ungoverned for a year. Real-time blocking is the actual requirement. Logging that an injection happened after the agent already acted on it amounts to a postmortem rather than a mitigation.
Observability across the full AI estate
The LLM observability platform market is projected to grow from $1.97 billion in 2025 to $2.69 billion in 2026, a 36.3% compound growth rate, reaching $9.26 billion by 2030. A market growing that fast is itself the evidence: visibility is a widely felt problem, not a niche concern for a handful of large banks.
Tools like Datadog, Langfuse, and Fiddler watch AI behavior without governing it. The control plane's observability layer is different because it sits on a stream of traffic it already controls, instead of watching from outside the fence. Full-stack observability here means cost attribution broken down by team, model, vendor, and use case, not one aggregated number nobody can act on. It means usage telemetry: which tools get called, how often, by which agent, the baseline that makes an anomaly actually look like one. It means audit logs immutable enough to satisfy SOC 2, HIPAA, and GDPR, with a per-request record of what an agent did and touched. And it means trace correlation, stitching a multi-step agentic workflow into one execution timeline instead of forcing an engineer to piece it together from a dozen server logs at two in the morning.
This closes the loop the MIT figure from the intro points at directly. Observability is how a company tells the difference between a pilot generating real value and one quietly padding the pile of AI spend with nothing to show for it. None of it matters if it lives in a dashboard nobody opens. It has to feed the SIEM and cost tooling teams already check every day.
How the five layers fit together in a reference architecture
None of these five layers stands alone. Each depends on the one underneath it, in a fairly strict order.
Identity is the foundation. Without a verified identity for every agent and tool, policy has nothing to bind to, and threat detection has no subject to attribute an event to. Policy sits directly on top of identity; a rule without an identity to enforce it against is just a sentence in a wiki somewhere. The MCP gateway is where identity and policy stop being abstractions and start being operational, intercepting every tool call and applying both in real time. Threat detection runs on that same traffic stream the gateway already controls, inline, on the same data, rather than as a separate inspection path bolted on the side. Observability is the output all four layers produce together: every authenticated, policy-checked, threat-scanned interaction leaves a record, and the audit log is really just the byproduct of the rest of the architecture doing its job correctly.
Deloitte's framing bears repeating, because it explains why a folder of config files can't substitute for this: the control plane is event-driven, coordinating a workflow as conditions shift, rather than a static service someone sets up once and forgets. Sovereign AI requirements plug in at the policy layer specifically. Jurisdiction rules and data residency constraints are policy configurations, not a separate system bolted on afterward. The reference architecture stays vendor-agnostic at each individual layer, but demands integration across all five. The real risk with point solutions is nailing one layer beautifully while leaving a gap between it and the next.
GitHub's public preview at Universe 2025 shows the full stack running inside a developer platform, governance and audit logging and enterprise controls sitting alongside the tools developers already use every day. Speakeasy's architecture, connecting every agent and MCP server through one governed layer, RBAC via Okta and Entra ID, real-time PII and injection blocking, cost and usage telemetry, is a concrete version of this same model, built for workforce-scale deployment.
What implementation actually requires from platform and security teams
This is infrastructure, requiring the operational rigor of a service mesh or an API gateway: deployment, maintenance, integration testing, and policy review that never really stops.
Start with inventory, before architecture. A team that doesn't know what AI tools and agents are already running inside the company can't write a policy that means anything, and it certainly can't configure threat detection against threats it doesn't know exist. Identity provider integration comes next, and it isn't optional; tool-level RBAC is flatly impossible without a connected identity store behind it.
The MCP gateway decision splits three ways: a managed service such as Amazon Bedrock AgentCore Gateway or Kong AI Gateway, an open-source, self-hosted option like Microsoft's MCP Gateway or Bifrost, or a purpose-built enterprise platform like Speakeasy. Each comes with a different operational footprint and a different set of compliance trade-offs, and the right answer depends on what a given security team is already staffed to run.
Whichever path a team picks, the evaluation criteria hold steady: OAuth 2.1 support in line with the June 2025 MCP specification revision, RBAC enforced at the tool level rather than just the application layer, immutable audit logs that export cleanly for SOC 2, HIPAA, and GDPR, and integration into the SIEM and cost tooling the security team already lives in every day. Get those four right, and the rest of the architecture has somewhere solid to stand.
