How agents are graded today
Most evaluation platforms are debugging tools in disguise. They let you judge your AI agent by observing its trace: the series of tool calls and reasoning steps it took to get to the final result. Imagine confirming a package was delivered by reading the driver's logbook instead of checking your doorstep.
For applications in regulated industries missteps carry heavy consequences, and inferring business outcomes from the agent's own testimony is insufficient—and perhaps negligent.
We advocate a more foolproof approach. Agents provide value because they act on business systems, altering their environment. Therefore, a standard for performance is simple to state:
An agent is correct when the world it leaves behind is the world the business requires.
Yet, the majority of AI agents are still judged by their own account of success. Why?
Trace-based evaluation
Most of the evaluation platforms started out as LLM grading tools. They filled the need to judge text output at scale—something the industry was not accustomed to. For simple LLM calls (e.g. question - answer) the model's response is the only artefact to score. Therefore, output-only grading is the only option.
Providing LLMs with tool access, i.e. allowing them to trigger code execution outside their context window, increases their usefulness. This, however, enlarges their impact surface. Now the LLM output (its reasoning and response) is only part of the complete outcome. Every part of the system the agent touches becomes an artefact that needs to be graded.
Luckily, the argument goes, the agent tells us which tools it called—and why—as part of its response. Instead of checking all the systems the agent possibly touched, we can fall back to grading the model's output only. We reconstruct the final state of the world by trusting the tool calls the agent said it made.
However, observing traces as a proxy for how the world changed comes with shortcomings. Consider, for instance, an agent in charge of refunds:
1. A trace captures claims about the world, not a verified state
The agent decides to refund a customer, calls the corresponding tools and considers it done. However, due to a malformed argument, downstream access denial, or hallucinated API response, the refund didn't actually go through. Yet, as far as the agent is concerned the world changed, even when it did not. It doesn't know, and neither do you.
2. The trace ends where the world begins
Even a perfectly faithful trace stops at the tool boundary; the world keeps changing beyond it. The refund tool execution triggers a ledger reconciliation, a customer notification, a web hook into accounting. All systems outside the agent's control, and none appear in the record.
Correctness itself lives outside the trace too: the identical sequence of calls is a correct refund for an eligible order and a policy breach for an ineligible one. To grade the trace, you must reconstruct the world anyway.
3. Grading traces is ambiguous and fragile
The same business outcome can be achieved in different ways. Grading the trace is grading the how, while business defines the what.
Many valid paths lead to the same refund: check the order first or the policy first, batch the lookups or not. A reference trace is counterproductive—cumbersome to write, unfamiliar to the business expert, and punishing to the agent's ingenuity.
Even if a single correct trace is defined, any change to the underlying system risks making the painstaking work obsolete. New prompt, new tools, new models. They can all cause the agent to find new ways to get the same outcome through different means.
From a business point of view, these shortcomings are disqualifying. For developers, however, trace-based evaluation remains ideal for debugging the system. It answers the question 'Where did the agent fail?'. But it is a poor proxy to get assurance on 'Did the agent get the right result?'.
Outcome-driven evaluation
Outcome-driven evaluation answers this question by comparing the world the agent left behind with the one business desires. This approach is technically more involved, yet conceptually much simpler than trace-based evaluation. It is the only way to get full visibility on the end-to-end behaviour, and decomposes into three steps:
- Seed the world in a deterministic way. Set up a controlled environment that mirrors the underlying business systems—a digital twin of the process the agent automates.
- Provide the agent with an input that triggers a business workflow: a refund request arrives in the mirrored mailbox.
- Observe the final state the agent leaves behind, and compare it with the desired state. The difference sets the grade.
In a nutshell: create the world you want, let the agent change it, compare the change with what the business requires. This approach is more aligned with business reasoning, and comes with key benefits:
1. The grade covers reality, not testimony
To judge the agent's behaviour we look at the final state. We don't rely on the agent's claims of what happened. We're not asking the agent whether the customer was refunded; we look at the ledger.
2. The grade captures the full business outcome
The digital twin contains everything the process touches: ledger, outbox, downstream systems. The comparison captures precisely the effects the trace cannot see, and judges them in the context that decides correctness: was this refund eligible?
3. The grade captures the what and not the how
Comparing two states of the world ends in pass or fail: either the agent changed the world as the business required, or it did not. Evaluation becomes a binary gate. And because only the outcome is graded, the suite is robust against every change of underlying system—new prompt, new model, new framework; it could be a human. Manifestly implementation-agnostic.
And when the agent fails a test, the evaluation hands the problem back to developers in their own language: the difference between desired and actual state is a fully specified, repeatable scenario to take straight into their (excellent, trace-centric) debugging tools.
AI agents will permeate the business world. However, current evaluation platforms leave risk owners deciding on incomplete and flaky evidence. Outcome-driven evaluation is the only way to get assurance: don't infer—just look.