← Back to concept explainers

Concept Fable

Trajectory evaluation

Conclusion first:Trajectory evaluation is the holistic assessment of an Agent's execution process, including plans, reasoning steps, tool calls, observations, retries, exception handling, and final output. It suits multi-step tasks because the final answer may conceal overreach, lucky guesses, incorrect tool calls, or non-reproducible paths. An enterprise quality loop should combine trajectory with trace, score by step, risk, and outcome, and distinguish correct failure, dangerous success, and truly reliable success.

Trajectory Evaluation · Evaluation

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

Only after seeing the footprints do you know why the hunter came back empty-handed

A hunter went into the mountains to chase deer and came back empty-handed at dusk. The village chief looked only at the result and judged him a failure. The hunter bowed his head and said nothing, and the others also assumed he had been lazy all day.

The old hunter did not rush to a conclusion, but took people to look at the mountain road. On the snow there were footprints, broken branches, a stream temporarily detoured around, and one place where something had lingered for a long time.

On the first inspection, he looked only at the last set of footprints. That section indeed led toward the empty valley, as if the hunter had taken the wrong path. But looking further back, everyone discovered that the main road had been blocked by a rockslide, and taking a detour was his only choice.

The village chief then asked, since the detour was reasonable, why still no deer had been hunted. The old hunter continued following the tracks and found that after the hunter caught up with fresh hoofprints on the mountainside, he mistook sheep tracks for deer tracks, and then spent half a day walking in the wrong direction.

If you only look at whether there was prey in the end, both cases are called failure: one is a correct response, the other a key misjudgment. If punishment or reward only watches the endpoint, you cannot learn what to change.

Later, the village evaluated hunts not only by what was brought back, but also by observation, choices, detours, waiting, and error correction along the way. Footprints can show how judgment formed, and can also expose which step led the rest astray.

The hunter is still accountable for the outcome, but the outcome is no longer the only evidence. To evaluate a complex action, you must look at the path it took: which decisions were reasonable, which deviations were fixable, and where it was merely luck.

Later the village training changed too. The old hunter would pick out a certain step and have the young people walk it again: if the stream had been rising at that moment, what other route was there; if this hoofprint had been seen, how would you confirm it was not a sheep. Evaluating the path is not about excusing failure, but about breaking the result back down into learnable choices. The village chief later admitted that looking only at venison was easiest, but also most likely to teach the wrong lesson. Good luck can mask bad judgment, and bad weather can hide good choices. Only by seeing the footprints clearly can you know whether to praise caution, correct a misjudgment, or admit it was just bad luck. Later the young hunters were no longer afraid to show their footprints, because footprints let training land on a specific step. This saying was later written into the newcomers' handbook, reminding everyone to look first at the on-site cost before talking about saving effort. And precisely because of that reminder, when similar difficulties came up later, the group did not rush to apply the old method. They learned to first ask which step bore the risk, then decide whether to let go, tighten up, or rearrange the process. This experience seemed plain, yet it made later choices much less vague and less dependent on luck.

Reveal

This story is about: trajectory evaluation

Trajectory evaluation is the holistic assessment of an Agent's execution process, including plans, reasoning steps, tool calls, observations, retries, exception handling, and final output. It suits multi-step tasks because the final answer may conceal overreach, lucky guesses, incorrect tool calls, or non-reproducible paths. An enterprise quality loop should combine trajectory with trace, score by step, risk, and outcome, and distinguish correct failure, dangerous success, and truly reliable success.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • The hunter's complete journey: Agent trajectory
  • The prey in the basket: the final output or task result
  • The Forbidden Forest and the cut rope marker: unauthorized, non-compliant, or dangerous intermediate actions
  • Footprint maps and knot markers: traces, spans, tool call logs, and observation records
  • Stop pursuit before the cub: correct refusal or safe termination
  • Dangerous shortcut brings back prey: the result succeeds but the process is unacceptable
  • Correct reading of the wind direction and tool selection: locate trainable parts based on trajectories
  • Final insight: Trajectory Evaluation evaluates how the Agent reaches the result

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

traceevalagent path

Related distinctions

This concept is easily confused with adjacent concepts; it is recommended to continue to the comparison page.

京ICP备2026014272号