← Back to concept explainers

Concept Fable

Root cause analysis

Conclusion first:Root Cause Analysis (RCA) is an analytical method for finding the true cause behind a problem, rather than only treating surface symptoms. It is commonly used in post-mortems for incidents, quality issues, system failures and business anomalies. For Agent systems, RCA helps teams determine whether the problem comes from the model, prompts, retrieval, tools, data, permissions, process design, or missing monitoring and governance.

Root Cause Analysis, RCA · Operations

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The well water is always muddy, yet the village chief keeps changing buckets

The well water in White Deer Village suddenly turned murky. The first to notice was the old tofu seller. He drew up the water, and the bottom of the bucket was full of fine sand. The village chief had that bucket dumped, swapped in a new bucket, and drew another. The water in the new bucket was still murky.

Some said it was the water drawer's heavy hand stirring up the bottom of the well. So they switched to a young person with a lighter hand. The water was still murky. Others said the bucket was too old and wood chips were falling into the water. So they bought a new bucket. The water was still murky.

Over the next few days, the village kept addressing surface problems: changing buckets, changing ropes, changing the water-drawing times, telling people to slow down. Each fix had some merit and kept everyone busy. But the tofu elder said the water does not turn cloudy in the bucket; it is already cloudy when it comes out of the well.

Only then did the village chief call in the old well builder. The well builder did not look at the bucket first, but investigated down along the well wall. He found a fine crack in the third layer of the well wall, and muddy water after rain was seeping in from there. The bucket, the rope, and the water drawer were not the root cause; they were just the places most easily seen after the problem appeared.

The well digger patched the cracks and also built a drainage ditch at the well mouth. The next day, the water slowly cleared. The village chief asked, why did all the many things we did before not work? The well digger said, because you kept dealing with the thing closest to your eyes after the water became muddy. The real problem was hidden before the water turned muddy.

Later the village set a rule: when a problem arises, first ask why five times. Why is the water murky? Because sediment entered the well. Why did sediment enter the well? Because the well wall cracked. Why did the well wall crack? Because rainwater washed it over time. Why did rainwater wash it? Because the wellhead drainage ditch was blocked. Why did no one notice? Because inspections only looked at the bucket and rope, not the well wall.

The village chief also required checking the consequences after every repair. Three days after patching the crack, the water cleared, showing the direction was right; if the water was still murky, they should continue investigating upstream rather than declaring that someone had not been careful enough. Root causes must withstand verification by results. Later, the village encountered sour tofu. Someone wanted to change the beans immediately, but the village chief first had everyone check according to the new rules: water, heat, stone mill, storage time, weather. In the end, it turned out to be the wooden rack used for overnight storage being close to stove ash. Only then did the villagers believe that root cause analysis was not just for well water, but a habit of tracing back to before the change occurred.

From then on, White Deer Village no longer treated busyness as a solution. Changing buckets can be fast; repairing the well is what actually helps.

Reveal

This story is about: root cause analysis

Root Cause Analysis (RCA) is an analytical method for finding the true cause behind a problem, rather than only treating surface symptoms. It is commonly used in post-mortems for incidents, quality issues, system failures and business anomalies. For Agent systems, RCA helps teams determine whether the problem comes from the model, prompts, retrieval, tools, data, permissions, process design, or missing monitoring and governance.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Muddy water: erroneous output, failures, or abnormal results seen by users
  • Changing the bucket, the rope, the person: only addressing surface symptoms or the most visible link
  • The water comes out of the well already muddy: the source of the problem is further upstream or lower down
  • Cracks in the well wall: the true root cause, such as data contamination, permission flaws, incorrect tool parameters or process loopholes
  • Fix the drainage ditch: not only repair the current problem, but eliminate the conditions that cause the root cause to recur
  • Five Whys: a common RCA questioning method used to approach the root cause layer by layer
  • Inspection only looks at buckets and ropes: the wrong scope for monitoring and checks can itself be part of the root cause
  • Swapping buckets can be fast, but fixing the well is what helps: the difference between temporary patching and root-cause repair

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

incidentobservability
京ICP备2026014272号