Concept Contrast Fable

Whether tool calls are accurate and whether tool calls are fast

Conclusion first:Tool-calling accuracy measures whether an agent chooses the right tool and uses the right parameters, while latency measures how long each call takes. The two often conflict: tool choices that spend more time reasoning tend to be more accurate but add response latency.

Tool-calling accuracy measures whether an agent chooses the right tool and uses the right parameters, while latency measures how long each call takes. The two often conflict: tool choices that spend more time reasoning tend to be more accurate but add response latency.

Fable

Two waiters, one fast and right, one fast and wrong

On South Street in the county town there was a restaurant with a courtyard separating the front and back halls. The proprietor hired two waiters. Every time Old Zhao took an order, he first stood still in the courtyard, silently recited the menu, then asked the proprietor about today's ingredients, and only then went to the kitchen to fetch the dishes. Old Qian was different: he rushed to the kitchen as soon as he took an order, thinking as he walked, and before he had finished reciting the menu he was already coming out with the plates.

In the first month, Old Zhao never delivered a dish to the wrong table, and customers praised him by name. Old Qian's orders were occasionally served wrong—once he sent the neighboring table's drunken shrimp to an elderly diner with dietary restrictions, nearly causing an incident. But Old Zhao's dishes came out slowly, and at peak times customers had to wait nearly half an hour.

The manager began studying the data. He found that Old Zhao spent on average three cups of tea per order, while Old Qian spent one cup. Old Qian could run thirty percent more orders a day, but if you deducted the redos by the proportion of wrong deliveries, the time cost of those orders plus the time to remake the meal actually saved little.

One afternoon, the shopkeeper put Old Qian in charge of cold dishes and Old Zhao in charge of hot dishes. Cold dishes had fewer varieties and were easy to recognize, so Old Qian's speed advantage was obvious; hot dishes required careful timing and complex prep, so Old Zhao's caution was more valuable. That day's table turnover was the highest of the previous month, and customer complaints were the fewest.

But the owner soon discovered a new problem. The restaurant took delivery orders, and riders waited outside. Old Zhao's three tea-breaks became the riders' nightmare—they reported the system's food-ready time to customers, who thought it would be five minutes, but ended up waiting fifteen. The riders' performance suffered from bad ratings, and they began refusing orders from this restaurant.

The shopkeeper took Old Zhao aside and asked if he could be faster. Old Zhao said the dishes would not be delivered wrong, but he could not guarantee speed. The shopkeeper then asked Old Qian if he could be slower. Old Qian said if he slowed down he would not be Old Qian anymore—he was smooth when running, and slowing down actually made him distracted and prone to mistakes.

Later the restaurant adopted a number-calling system, so guests knew when they ordered how long their dish would take. The shopkeeper set up Old Zhao as a dedicated channel for slow orders and Old Qian as the fast-order channel, separating them at the system level rather than having the two servers compete on the same path. Accuracy and speed are not things one server can solve at the same time; the system needs to separate them at the entrance.

Later the manager posted this routing table at the serving window, and new waiters first looked at the dish's risk before deciding how fast to step.

Tool call accuracy

Measures whether the agent selected the right tools and passed in the right parameters. High accuracy means tool use matches expectations, but it usually requires more reasoning time.

Tool call latency

Measures the average time an agent needs to execute one tool call. Low latency means fast response, but too little reasoning depth may lead to incorrect tool selection.

Story Mapping

Old Zhao recites silently before taking ingredients: reasoning fully before a tool call improves accuracy but adds latency
Old Qian rushes as soon as he takes an order: fewer reasoning steps reduce latency, but the tool selection error rate rises
Cold dishes to Old Qian, hot dishes to Old Zhao: route by task complexity—simple tasks use low latency, complex tasks use high accuracy
Delivery rider waiting for Old Zhao: in low-latency scenarios, high accuracy becomes the experience bottleneck instead
Old Qian makes mistakes when he slows down: in some task types, fast execution and accurate execution are inherently contradictory
Use a queuing system for intake triage: separate different optimization objectives at the system level instead of having a single agent handle them all at once

Moving to Agent performance optimization

When optimizing tool calls, first clarify whether your scenario fears errors more or slowness more. Scenarios that tolerate errors and can be corrected downstream, such as customer service conversations and document generation, can lower latency and accept a certain error rate; scenarios where errors are costly, such as financial trading and medical analysis, must prioritize accuracy even if responses are slower. The most effective approach is to route at the entrance: simple, highly repetitive calls go through the fast channel, while complex, first-time scenarios go through the accurate channel. Do not let the same Agent run on both paths at once.

Tool call accuracy Tool call latency