AI observability: Why traditional monitoring isn't enough for AI systems
Traditional monitoring watches four things: whether the system is up (availability), how fast it responds (latency), how often it errors (error rate), and how hard the underlying resources are working (resource utilization).
This system works because software is deterministic: the same input produces the same output, so a deviation means something broke, and it announces itself as a timeout.
However, AI systems are non-deterministic. Ask the same question twice and you'll get two different answers, both valid. There's no baseline to deviate from, so "Is this response normal?" stops being a threshold check and starts becoming a question of where this answer falls among many valid ones. Worse, an answer that's wrong can look identical to an answer that's right. The failure lives in the content, and the content is not a signal your monitoring stack collects.
When you call a hosted model, your dependency can change without you deploying anything. The provider updates the model and your system behaves differently on Tuesday than it did on Monday. Every change-correlation workflow an IT operations team relies on starts with something to correlate against. With hosted models, there isn't anything.
None of that is hypothetical. Amazon's retail website went down for roughly six hours in early March 2026, with more than 22,000 users reporting failures across nearly every feature. Internal documents linked that week's incidents to AI-assisted coding errors. Fortune later reported the chain of events: an internal wiki page was out of date, an AI agent read it and drew a conclusion. An engineer asked the agent a question, got that same conclusion, and shipped a change based on it. Amazon's position was that AI never wrote bad code—an engineer followed advice an agent inferred from a stale page. Every monitor Amazon ran performed correctly the whole time, because nothing in that chain was something a monitor watches.
None of this means monitoring is obsolete. Uptime, latency, error rate, and utilization still matter. They're just no longer the only questions worth asking.
The AI failures that pass every health check
If your monitoring only tells you the system responded, there are four things it isn't telling you.
Was the answer correct?
A customer asks about returns and the model quotes a policy the company has never had. An analyst asks about a filing requirement and gets a regulation that doesn't exist. Every signal your monitoring collects says the system worked, because by the only definition monitoring understands, it did. There is no status code for fluent and confidently false.
What did it actually do?
When a chatbot invents something, the damage stops at the text on the screen. When an agent invents something, it moves to the next step of the plan. That's what happened at Amazon. The inference was never visible to any metric Amazon monitors, and by the time the consequence was visible, it was an outage rather than an error.
How many answers were actually wrong?
A model that's right 99.9% of the time looks flawless in a demo and in every test you'll run before launch. Put it in production at 10,000 requests a month and that same rate produces 10 wrong answers, each one indistinguishable from the other 9,990 in your logs. Ultimately, nothing about the failure rate changed, only the volume did.
How much did that cost?
Say a task normally costs you three cents. An agent hits a loop, retries 11 times, and that task now costs four dollars. Throughput looks healthy, because from the infrastructure's point of view the system is doing exactly what it was asked to do, repeatedly. The Open Worldwide Application Security Project (OWASP) tracks this as unbounded consumption in its 2026 LLM Top 10, and it's the clearest case of a metric your monitoring dashboards don't treat as an operational signal.
What AI observability adds to the stack you already run
So does this mean you need a whole new monitoring stack? No. You need your existing stack to start collecting four things it doesn't collect today.
Let's start at the trace. A conventional trace shows you the hops between services. An AI trace shows you the steps inside the decision: which documents came back from retrieval, which tool the agent picked, what it passed to that tool, what came back, and what it did next. That's the record that would have shown someone the stale wiki page before the change shipped.
Traces tell you what happened. They don't tell you if it was right.
You can read every step an agent took and still not know that the outcome was wrong. So quality has to become a signal you watch: is this answer supported by the documents it cited, does it address what was asked, and does it stay inside policy? You define what correct means for a given system, then you monitor that definition with a threshold and an alert behind it.
That says nothing about the 11 attempts it took to get there. An agent can loop, retry, burn four dollars on a task that normally costs three cents, and still deliver a perfectly good answer at the end; so cost has to be attached to individual requests, users, and workflows. Otherwise you find out the inefficiency in 30 days when you get the bill.
And all of that telemetry is built to catch problems while they're happening. Proving what happened four months ago is a different job, and most monitoring stacks are not equipped to do it. Detailed logs age out, and by the time a customer disputes a decision, the detail that would have answered it is gone. A record that survives that gap holds more than a timestamp and an outcome: it holds the input that came in, the output that went out, the tool calls in between, and which model produced them. Retention is a deliberate decision, with a defensible reason for every option picked.
ManageEngine Applications Manager monitors Amazon Bedrock workloads on exactly that layer: tracking invocations, latency, time to first token, token consumption, quota usage, and throttling, along with the configuration, knowledge bases, and guardrails attached to Bedrock Agents. Those signals sit next to the servers, databases, containers, and APIs the same service depends on, all in the same console.
The compliance clock nobody reset
The record-keeping piece is the one with a deadline, and the deadline is not the one most coverage reported.
What actually changed in July?
The European Union AI Act's heaviest requirements—the ones covering high-risk systems—were set to land on Aug. 2, 2026. That deadline came with a problem: the technical standards defining how to actually comply with it weren't finished. So on July 24, 2026, eight days before the deadline, the EU published Regulation (EU) 2026/1744, the Digital Omnibus on AI, and pushed the high-risk obligations out. Standalone systems now have until Dec. 2, 2027. AI built into regulated products has until Aug. 2, 2028.
What didn't change?
Two things. Article 50, the transparency requirement, went live on Aug. 2, 2026 as originally planned, and reaches systems already on the market on Dec. 2, 2026. And Article 12—record-keeping and logging—wasn't amended at all.
What changed was when you had to demonstrate compliance. What didn't change is what you have to demonstrate it with.
Why can't you postpone any longer?
The deadline moved. The evidence requirement didn't. Because logging is the one requirement you can't do retroactively. You can write a policy in a week. You can commission an audit in a month. You cannot produce a record of a decision your system made last March if nothing was capturing it last March.
The 16 months are the window in which the evidence either gets captured or doesn't. A team that starts building its logging in late 2027 will hit the deadline with a compliant process and a year and a half of nothing behind it. And this gap will be permanent, because the only thing that could have filled it was telemetry nobody turned on.
Where this is going
Gartner® expects 40% of organizations deploying AI to be running dedicated AI observability tools by 2028, watching model performance, bias, and outputs. Gartner analyst Padraig Byrne framed the reason plainly: that visibility gap makes scaling risky.
Internal documents reported by the Financial Times show Amazon requiring senior engineers to sign off on AI-assisted changes. More oversight, not less AI. That's the shape of what works. The teams who get this right will be the ones who stop asking their monitoring whether the system responded and start asking it what the system did.