Real-Time Observability Telemetry in an AI Control Plane

A container either runs or it doesn't, and a query returns or it times out. An agent, given the same prompt twice, might call a different tool, touch a different database, and produce a different downstream effect each time, and your existing logs will tell you almost nothing about why. That's the mismatch: deterministic infrastructure trying to supervise non-deterministic software. Observability has to be the operational core of any serious AI control plane from day one, before the first incident report ever gets written, not after.

Consider what a single agent session can do without anyone noticing. It reads from a production database, calls a third-party API, writes the result into a SaaS tool, and does all three back to back with no human anywhere in the loop. Your APM tool will happily confirm the container stayed up and the API call returned a 200. It won't tell you what the agent decided, what it actually touched, or whether some document it ingested mid-task quietly rewrote its instructions. Knowing a service is running was never the same as knowing what it did with the authority you handed it. With agents, that gap stops being an inconvenience and starts being a governance failure with a start date and no end date.

What the AI control plane is and where observability fits inside it

An AI control plane keeps inventory of every AI system in the enterprise, watches what it does, enforces policy against it, and holds a record for later audit. Kubernetes gave engineering teams a control plane for container orchestration. The AI control plane does the same job for agents, LLM calls, MCP tool connections, and the growing volume of agent-to-agent chatter most security teams currently can't see at all. Forrester named "agent control plane" its own emerging market category in late 2025, which tells you this stopped being a side project and became infrastructure with real stakes attached.

Four layers do the actual work. Access and identity decide who, or which agent, gets to connect and with what authority; policy enforcement decides what actions are allowed and under what conditions; threat detection catches prompt injection, data leakage, and behavior that breaks from the norm. Observability sits underneath all three as the telemetry layer, the thing that makes the other three mean anything.

Access policy without telemetry is a document you wrote once and hope holds; threat detection without telemetry has no baseline, so it can't tell normal from suspicious even in principle; and cost management without telemetry is a bill you get surprised by every month, forever. ISG Research expects most large enterprises to have some form of cross-platform control plane for autonomous systems in place by the late 2020s. The category is moving fast because the underlying problem, agents acting with real authority and no consistent record of what they did with it, isn't fixing itself.

How agent scale makes ad-hoc monitoring untenable

Not long ago, most enterprises ran a handful of LLM integrations that a couple of engineers tracked by memory and a shared spreadsheet. That era is over, and it's not coming back. Enterprises now run dozens of models, hundreds of AI-powered applications, and agent fleets that grow every quarter without much ceremony. Gartner projects Fortune 500 companies will be running enormous numbers of AI agents by the end of the decade, up from close to none just a few years back. That's a step change compressed into a single planning cycle.

Each agent is a non-human identity holding access to more than one connected system at once, capable of moving substantial volumes of data per session. The blast radius of one misconfigured or compromised agent scales directly with how much access somebody handed it, so the cost of getting agent permissions wrong rises every time a new one gets deployed.

Ad-hoc monitoring worked fine back when there were three integrations and one engineer who could hold all of them in his head. It breaks the moment every business unit starts standing up agents against production systems. The operational question stops being "is this agent running?" and becomes something genuinely harder: what is every agent in this organization doing right now, against what data, under whose authority? Tailing logs won't answer that question; the old tools were never built to map onto what agents actually do. What's needed instead is a telemetry architecture built for multi-system, autonomous workflows.

What enterprises must actually instrument to govern agents

Not all telemetry carries equal weight, and this trips people up constantly. Logging the raw prompt an agent received is low signal; it tells you what someone asked for, not what happened next. The real signal lives in runtime behavior, the actual sequence of actions an agent took once it got moving.

A few categories matter most. Tool- and function-call sequences show what the agent invoked and in what order, which is how you spot a workflow deviating from its usual shape; records accessed per session capture volume and sensitivity of data touched, not just whether a connection succeeded; and identity and effective authority tell you which human the agent acts on behalf of and what that human's permissions actually cover across every connected system, which is often far broader than anyone realizes until an agent goes and tests the boundary. Agent-to-agent communication needs to stay traceable end to end, so when one agent delegates to a sub-agent, the chain of authority stays visible instead of vanishing into a black box. MCP tool call logs need to capture every invocation through the gateway with full context on which agent made the call and what it touched.

Cost telemetry gets treated like a finance afterthought when it should be a first-class operational signal. Token consumption per agent, per workflow, per team is the only real basis for chargeback, capacity planning, and catching an agent quietly burning through budget in a loop nobody noticed for three days. Without it, a platform team staring at a cloud bill has no way to tell a high-value workflow from a runaway one; the invoice looks the same either way.

Instrument to OpenTelemetry-compatible signals if you want to avoid vendor lock-in, since that lets telemetry flow into whatever SIEM, observability platform, or warehouse you already run. Request and response logs don't capture an agent's decision sequence, and infrastructure uptime metrics don't tell you a thing about what data got touched. Pretending either one substitutes for the instrumentation above is how teams end up governing agents with tools built for a different problem.

Observability as the mechanism that makes access policy enforceable

Write an access policy with no way to check whether it's being followed, and what you have is a declaration, not enforcement. You can define, on paper, that a given role should never touch a certain dataset and still have zero idea whether that boundary held in practice. Enterprises routinely have agents running in production across multiple business units with no formal strategy yet for managing the non-human identities behind them. That gap is not small, and it is not unusual.

Telemetry is what closes it operationally. Every agent request should carry an identity token, and the telemetry layer's job is recording whether the action taken lines up with what that identity is actually allowed to do. Drift shows up in the pattern over time: an agent nudging against the edge of its granted scope, or occasionally stepping past it, leaves a trail that telemetry can surface before it becomes an actual breach instead of after.

On-behalf-of identity propagation matters here too. When an agent acts for a specific user through OAuth 2.1 or OIDC, telemetry can attribute every downstream action back to that human principal instead of losing it in a shared service account where nobody owns anything. Entra Agent ID and Okta for AI Agents have both published formal blueprints for agent identity, and those blueprints are genuinely useful, but they're only as good as the telemetry confirming agents actually stay inside them. RBAC enforced at the protocol layer beats hoping a prompt behaves itself, but it only works with a feedback loop telling the policy engine whether enforcement is holding up.

How real-time telemetry enables threat detection that works at agent speed

Prompt injection tops most lists of AI-specific vulnerabilities for a simple reason: the model can't tell the difference between an instruction from its operator and one smuggled in through a document it just read. Detection has to happen outside the model, because the model isn't built to catch it and never will be.

Injected instructions hide in a document, an email, a web page an agent ingests mid-workflow, and by the time a human opens a log file, the action that injection triggered may already be finished. Agents are also frequently granted far more access than the job requires, so even a clumsy injection attempt can yield damage way out of proportion to whatever risk anyone thought they were signing up for.

Real-time telemetry catches what static rules miss entirely. A behavioral baseline, built from what a normal tool-call sequence looks like for a given workflow, flags deviations the moment they happen rather than a week later; sequence anomaly detection catches an agent suddenly pulling records or calling tools well outside its usual pattern and fires an alert before anything leaves the environment; and real-time inspection of data in transit catches PII and secrets moving through a session that were never supposed to be exposed, which a config file review would never surface on its own.

OWASP's System Prompt Leakage entry, classified as LLM07:2025, points at a related risk: a lot of exposure traces back to secrets and credentials embedded directly in prompts. Telemetry at the gateway layer catches and redacts that before it spreads. Speed is the whole game here; by the time a batch log gets reviewed, an agent may have already run through dozens of steps, so detection has to fire during execution, not three hours after.

Shadow AI makes everything worse, since tools operating outside the governed layer produce no telemetry at all, so there's nothing to check anomalies against. An incident on shadow AI takes far longer to catch than one on a monitored system, for the least mysterious reason imaginable: you can't observe what never left a trace.

Telemetry as the only reliable signal for managing AI cost at scale

AI compute cost doesn't behave like a fixed line item. It scales with agent activity, token consumption, and how many workflows are running simultaneously, and it can climb faster than anyone budgeted for. Without granular telemetry, a platform team can't trace cost back to a specific team, workflow, or agent. All they get is a total on an invoice, no breakdown, no story behind the number.

Good cost telemetry captures token consumption per agent session, broken out by model and workflow, plus cost allocation by team or department that can actually support a chargeback model. It also has to catch runaway agents: cost spikes pointing to a loop, excessive retries, or an agent operating well past the scope it was built for.

Cost observability doubles as a safety signal, which gets underappreciated. An agent burning tokens at multiples of its normal rate isn't just a budget line item gone sideways; it's a behavioral anomaly worth investigating on security grounds too. Chargeback and showback only work if the telemetry behind them is accurate. Without it, AI cost sits as a shared infrastructure expense nobody owns, and nobody owning the cost means nobody has much reason to govern how their agents behave. As fleets keep growing, cost management without telemetry stops being inconvenient and becomes flatly impossible. Instrumentation that makes every agent's consumption visible is the only path to AI economics that hold up past next quarter.

The architecture that makes observability continuous rather than retrospective

The gateway layer is the natural place to instrument, since every agent request, every tool call, every MCP invocation passes through it. Instrument once at that chokepoint and you get visibility into everything downstream, which beats instrumenting ten different services and hoping they agree with each other.

The gateway and the control plane solve different halves of the problem, and it's worth keeping them straight. The gateway enforces policy in real time as requests come through, while the control plane aggregates the telemetry that gateway produces, finds patterns in it, and updates policy in response. A gateway without a control plane gives you enforcement with no feedback loop, and a control plane without a gateway gives you visibility with no way to act on it. Both are needed, and neither one is optional.

MCP brings its own telemetry requirements that general-purpose logging just doesn't cover. Tool call logs need full context, showing which agent, which server, which tool, and which user identity sat behind the call, while audit trails need to survive across session boundaries, especially for workflows running long or spanning multiple steps. MCP-specific threats, tool poisoning, rug-pull attempts where a tool's behavior changes after it's already been trusted, cross-server data shadowing, are invisible without instrumentation built specifically to watch for them, and nobody stumbles onto a rug-pull by accident.

OpenTelemetry compatibility is the connective tissue: telemetry emitted to OTel-compatible endpoints flows straight into whatever SIEM or observability platform an enterprise already runs, no rearchitecting required. And here's the crux of the whole argument: batch log review doesn't move at the speed agents operate at, full stop. The architecture has to produce signals that trigger real-time alerts, well before any post-incident report gets written weeks later, after the damage is already old news.

Speakeasy's AI control plane is built around this exact architecture: a governed layer that surfaces cost and usage telemetry, catches shadow AI and prompt injection as they happen, and plugs into identity providers already in place. Every other governance function in that system has to run through observability to work at all.

What governance looks like when observability is the operational foundation

When telemetry runs live, security teams stop spending their days reconstructing what happened after an incident and start catching behavioral drift before it becomes one. That's a genuinely different posture, and it changes what "governance" means on a Tuesday afternoon, not just in the quarterly review deck.

Policy stops being something you set once and forget about. Access rules, rate limits, and cost thresholds adjust based on what telemetry is actually showing, instead of sitting frozen until the next scheduled review catches up with a reality that's already three months stale. Audit logs become a compliance asset in their own right. A full, attributable record of every tool call, every piece of data touched, every identity asserted, is exactly the evidence regulators and internal auditors ask for, and increasingly what AI Act compliance reviews will expect to see on the table.

The rollout model shifts too. With continuous observability in place, an enterprise can extend AI access to every employee from day one, because the governed layer is watching in real time rather than forcing security to manually review each use case for months, a process that was never going to scale to agent fleet sizes anyway. Shadow AI becomes solvable in a way it simply wasn't before. Once sanctioned traffic runs through the governed layer, anything outside it stands out immediately as a gap in the telemetry, not a mystery that takes a months-long investigation to untangle.

Take observability out of the picture and the rest of the control plane is theater. Leave it in, and it's the only reason any of the other pieces actually hold.