A healthy system can still be wrong 

Latency, response time, CPU usage, uptime. These still matter, but they won't tell you if an AI system is giving you useful or trustworthy answers. 

Santosh Kumar Perumal, Senior Cloud and DevOps Engineer at Tata Consultancy Services, put it plainly during a panel talk: traditional observability watches infrastructure and app performance. AI needs a wider lens. Using a retrieval-augmented generation system as his example, he said teams need to track which documents produced an answer, which model version, token use, response time, and cost per output. 

These aren't nice-to-haves. They're how you work out whether an answer can be trusted. 

Conventional software usually throws a visible error when it breaks. AI doesn't. It can keep producing convincing output while quietly drifting off course. One panellist summed this up:  

"With AI, you'll still get data, but that data is no longer accurate." 

No outage. No error code. Just declining usefulness nobody notices. 
 

Monitoring AI means measuring at several levels 

Infrastructure and service health 

The baseline layer. Is the system stable, scalable, financially sustainable? Track availability, latency, GPU utilisation, failure rates, cost per query, token consumption, and uptime of external models. This tells you the service can run reliably. It says nothing about whether it's producing the right result. 

Model and output quality 

This is where it gets specific to AI: accuracy, hallucinations rates, relevance, consistency, drift, retrieval quality, prompt effectiveness, and human correction rates. The panel flagged semantic drift as the one to watch. Similar inputs shouldn't start producing wildly different outputs without someone knowing why. That means you need a baseline for expected behaviour, otherwise there's no way to tell when something's moved outside acceptable range. 

What "acceptable" looks like depends on the job: a customer service assistant lives or dies on resolution rates, a coding assistant on defect rates and security findings, a knowledge tool on retrieval accuracy and source attribution. 

 
Not every anomaly is an incident 

More data doesn't mean better observability. AI monitoring tools flag unusual behaviour constantly, but unusual isn't always a problem. 

Kumar warned that giving agents too much control over incident detection risks turning every anomaly into an incident, flooding SRE teams with alerts and recreating the exact alert fatigue AI was supposed to fix. 

Detection and judgement are two different things. AI can spot patterns, correlate signals, search logs, and suggest causes. It shouldn't be assumed every flagged change needs action. Organisations need to measure the monitoring itself: false positives, false negatives, alert volume, time to identify a real issue, percentage of AI recommendations engineers actually accept. 

Hundreds of technically correct but operationally irrelevant alerts isn’t observability. It’s just noise. 

 
AI changes what you need to trace 

Logs, metrics, traces - that’s the standard observability stack. AI adds more: the prompt, the context supplied, documents retrieved, model version, tools calls and their outputs, the final response, guardrails triggered, human overrides, actions taken afterward. 

One panellist put the goal simply: visibility not just into whether a tool call succeeded, but into what the agent is thinking and doing, so teams can audit is actions after the fact. 

The catch is most organisations already run fragmented observability, logs in one tool, metrics in another, traces somewhere else. Humans navigate that mess through institutional knowledge. As Stephen Crowley of Bronto explained in Telemetry in the Age of Agents, an AI agent doesn't have that context automatically. Someone needs to make it explicit, or the agent searches the wrong systems and draws incomplete conclusions. 

AI observability isn't just bolting an assistant onto existing dashboards. You must know where your operational data lives and whether an AI system can reach it safely. 

 
Speed has to be measured alongside risk 

Panellists predicted AI agents could help developers ship changes five to ten times faster. Good for output, bad for detection windows, faster shipping means less time to catch quality, reliability, and security issues before they land. 

Annual security reviews can't keep pace with an environment that changes daily. Risk needs monitoring at delivery speed: security findings per deployment, vulnerabilities in AI-generated code, exposed secrets, prompt-injection attempts, agents with excessive permissions, and tool calls outside expected workflow. 

Amir Shamiri of Mend.io reinforced this from the security side in Don't Send Your Agent to My LinkedIn. As agents get access to browsers, development environments, APIs and internal tools, the number of potential failure points grows. His principle: 

"Trust, but verify." 

That should shape how you might measure performance, not just whether a task got done, but how. An agent might “successfully” fix a connectivity issue by opening far more ports than necessary. Completion rate says success. Real observability catches the security holes underneath it. 

 
Human intervention is itself a metric 

How much humans step in tells you a lot about whether AI is actually helping. Worth tracking: percentage of outputs accepted unedited, override frequency, escalation rates, time spent reviewing AI output, errors caught in review, automated actions rejected before execution. 

Oversight should scale with risk, not blanket every action with the same approval process. Low-stakes dev environment tasks can run more freely. Anything touching customer data or production systems need tighter controls. 

Rakesh Rajendra, speaking in AI for SRE & CloudOps: Hype vs Reality in Production, said the industry made a mistake promising full autonomy too early. In this experience, AI is strong at investigation, triage and evidence-gathering, less strong at being left alone in mission-critical environments: 

“Human in the loop is mandatory.” 

His takeaway: use deterministic automation where you can, save AI for work that genuinely needs interpretation. Even good models get it wrong sometimes, and production can’t treat every confident answer as correct. 

 
Technical performance isn’t business performance 

Even solid technical measurement won’t prove AI is worth the spend. Boards want to know if it’s improving something they actually care about: revenue, cost, time saved, faster response, fewer errors, better decisions. 

The real is in connecting the two. Lower retrieval accuracy leads to more customer escalations. Slower response times kill adoption. Higher hallucination rates mean more manual review. Miss that connection and you end up optimising metrics that don’t move the business at all. 

 
Who owns AI performance? 

No single team can answer this alone. It splits across the business: infrastructure teams own availability and cost, data and AI teams own accuracy and drift, security owns access and misuse, product owns adoption and outcomes, legal owns explainability and compliance, leadership decides if it’s worth the investment. 

Before anything goes live, agree on what success looks like, which failures are unacceptable, who monitors what, and who can pull the plug. The real question isn’t who owns the model. It’s who owns each outcome it touches. 

 
From dashboards to decisions 

More dashboards isn’t the answer. Most organisations already have too many. Start with four questions: What was this system mean to improve? What does failure look like even while it’s technically running? What signals give early warning? Who has the authority to act on them? 

A system can be reliable but inaccurate. Accurate but too expensive. Efficient but unused. Popular but insecure. Profitable but impossible to audit. Deployment proves AI can run. Observability tells you whether it should keep running.