TL;DR: Many organisations now measure AI outputs for accuracy, relevance, and harmful content using automated evaluation techniques. While these provide valuable signals, they are still evolving and often vary by platform, making enterprise-wide consistency difficult. This blog explores Next-Gen Observability approaches to compare performance, quality, and risk across AI use cases.
Next-Gen Observability extends existing observability systems into AI operations, helping you achieve outcome alignment between AI and business strategies. It aims to create a shared structure that aligns different teams’ observability efforts over time and provides a big-picture view of AI impact.
In practice, AI-related metrics and traces across multiple services are grouped together under common themes, such as productivity, cost, or risk. This creates a shared view of AI performance across technology, GRC functions, and business units.
Why Next-Gen Observability Matters
Next-Gen Observability marks a transition in how enterprises manage AI. It creates the mechanism through which organisations ensure AI quality, manage risk, control cost, and build confidence that AI-driven decisions and outputs can be trusted at scale.
Building Compliant AI Systems
Next-Gen Observability reduces the manual effort typically required to maintain compliance in a fast-changing AI environment.
Next-Gen Observability systems can continuously compare regulatory obligations with internal policies and control frameworks. This allows teams to observe where regulatory expectations and internal policies diverge, and where engineering specifications failed to fully implement required controls.
Instead of modifying systems after they have been built, teams can actively shape better decisions before risk and cost are locked in.
Cost Control
AI adoption at scale has led to token usage costs significantly increasing. Integrating AI cost across usage and business outcomes allows leaders to quickly correlate the cost of change and the cost of run with the derived output value. They can also track relevant metrics across AI-impacted service flows to identify:
Areas where further automation adds most value.
AI behaviours or decision patterns that drive up costs.
AI impact vs. cost over time
Governance at scale
Next-Gen Observability becomes a mechanism for control and alignment. Signals from AI system behaviour can be used to enforce safety rules, policy constraints, and operational guardrails in real time. Organisations can ensure AI systems are operating within approved boundaries and align with business and regulatory expectations.
As agentic workflows manage more operational outcomes, organisations can manage AI systems with the same discipline, assurance, and confidence expected of any critical enterprise system.
Next-Gen Observability In Practice
What Next-Gen Observability looks like in practice is unique to every organisation. We give some examples below based on our work with clients.
AI-Driven Software Development
Software development is shifting from its early vibe-coding days to a more intentional, spec-driven approach.
Requirements → Detailed Specification → AI Generation → Validation
Spec-driven development changes where the most important decisions are made. Instead of risks emerging late in the delivery cycle, trade-offs and constraints are defined upfront in a shared specification that becomes the single source of truth.
Next-Gen Observability treats specifications as observable assets. Product, design, engineering, and risk teams collaborate on the same artefact, reducing misalignment before code generation itself. Providing feedback loops with recommendations and pull requests significantly improves quality and time-to-market.
Every decision, constraint, and change is versioned and traceable at the specification level, improving governance and auditability. Leaders gain visibility into why certain design choices were made, not just what was built.
RAG-Based AI
A retrieval-augmented generation (RAG) pipeline often feeds information from enterprise knowledge sources to AI models in common AI solutions such as chatbots, assistants, and AI agents.
Today, many RAG-based AI systems have reached a point where organisations can run them with reasonable confidence. Most enterprises can already monitor operational fundamentals such as response time, error rates, usage, and cost.
However, scaling them remains challenging. There is inconsistency across tools and teams when it comes to two critical aspects:
Transparency - Understanding why an AI behaves a certain way and
Impact - Whether it is delivering the intended business value.
Next Gen Observability can solve this by incorporating systematic evaluation and oversight from the start, rather than adding them after problems emerge. From a technical perspective, this can look like:
Trace each step of the RAG pipeline - from the initial user query, through document retrieval and ranking, to prompt construction and final response generation.
AI-assisted analysis so teams investigate issues using natural language, making troubleshooting faster and more accessible.
Validating retrieved information against source material and checking whether answers remain relevant and trustworthy
And more.
The goal is to create an end-to-end view that allows organisations to understand where failures or poor outcomes originate, whether from missing data, weak retrieval, or model behaviour.
Agentic AI
Agentic AI is the next iteration of AI development, in which autonomous AI systems work alongside humans to achieve organisational outcomes. They can automate entire business processes, working with a range of third-party tools and handing off tasks to each other so the system achieves a common goal.
In these systems, the challenge is no longer just whether an AI model responded correctly, but whether a chain of automated decisions behaved as intended across agents, tools, and systems. Understanding transparency and impact is even more difficult.
Next-Gen Observability can act as a control layer for agentic systems. It can provide visibility into how decisions were made, how agents interacted, and where automation may be looping, drifting, or introducing risk.
At its core, this approach creates hierarchical visibility. Leaders and operators trace outcomes from a user request, through the orchestration layer, down to individual agents and the tools they invoked. When incidents like excessive latency, cost overruns, or incorrect outcomes occur, teams can quickly pinpoint the root cause.
Summary
Area | Current maturity level (2026) | What is mature | What is still emerging |
GenAI / LLM apps | Some observability systems running in production | Infrastructure metrics, token/cost tracking, Natural language queries over data | Standard semantic schemas for prompts/results across tools. Alignment to Open Telemetry standards for outputs and metadata analysis |
RAG pipelines | Early scaling of observability systems | Pipeline traces, retrieval & answer metrics | Uniform, cross‑platform evaluation standards and adherence to compliance standards |
Agentic workflows & interoperability | Just getting started | Hierarchical traces, decision‑path analytics, protocol‑level tracing, and structured context flow | Industry‑wide patterns for KPIs, SLOs, and governance Broad adoption as a default control plane for observability |
Final words
Observability in the AI ecosystem is mixed and focuses only on performance and operational metrics. Challenges exist in extending observability into actionable insights to drive greater value and more effective alignment with company strategies.
There is a growing movement to address gaps through Next-Gen Observability:
Elevate existing observability from the technology domain to an enterprise capability with AI
Use this AI-powered observability system to maximise the impact of core AI initiatives.
