← Back to Concept Contrast

Concept Contrast Fable

Human in the loop and human on the loop

Conclusion first:One requires a person to stand at a checkpoint that every process must pass through, nodding before anything can proceed. The other requires a person to stand high and watch, reaching in only when the system goes off track. Choose the wrong design, and either the person becomes the system's bottleneck, or the system quietly makes mistakes when no one is watching.

One requires a person to stand at a checkpoint that every process must pass through, nodding before anything can proceed. The other requires a person to stand high and watch, reaching in only when the system goes off track. Choose the wrong design, and either the person becomes the system's bottleneck, or the system quietly makes mistakes when no one is watching.

Fable

The gatekeeper and the person on the watchtower

Old Sun's automated workshop has two human-machine collaboration models. The first production line is called the "approval line": each product goes from the first process to the last, passing through three manual inspections along the way. Inspectors stand beside the conveyor belt, glance at the product as it passes, and stamp it. Without a manual stamp, the product can't move to the next process.

The second production line is called the "monitoring line": the product flows automatically from start to finish, with no one stopping it midway. But on the second floor of the workshop there is an observation room, where a dispatcher sits in front of a status screen for the entire production line. As long as temperature, speed, and pass rate are within the set range, the dispatcher does nothing. Once an indicator deviates from the preset range, a red light lights up on the screen, and only then does the dispatcher intervene—pausing the line, adjusting parameters, or pulling the abnormal batch out for separate inspection.

The workshop took a large order and had to run two production lines at the same time. A month later, the factory manager compared the data from the two lines. The approval line's output was only forty percent of the monitoring line's, because every process step had to wait for a person to stamp it, and people have times for rest, distraction, and queuing, so the machines had to wait. But the approval line's defect rate was almost zero—every product had passed through human judgment.

The monitoring line had high output, but a problem arose: once a temperature sensor drifted by two degrees, within normal fluctuation range, and the system didn't alarm. After running continuously for three hours, a key dimension of the product was already out of tolerance, but the entire batch passed automated inspection. By the time the dispatcher noticed, a whole batch of defective products had been produced. The dispatcher admitted: during those three hours he had not actively spot-checked the product, just stared at the screen waiting for a red light.

The factory manager laid the experience of the two production lines on the table. The core of the approval line is that a person must say "yes" before release—the person is in the decision path at every step, which is called in the loop. The core of the monitoring line is that a person does not need to intervene at every step, but must patrol regularly and take over when something goes wrong—the person is in a supervisory position outside the loop, which is called on the loop.

The factory manager chose different modes for each scenario. For processes involving safety, compliance, and funding decisions, the approval line was used, with humans in the loop. For high-volume, controllable-risk processes in daily operations, the monitoring line was used, with humans on the loop, but a hard rule was added for dispatchers: they must go to the production line for a random spot check once every hour, and cannot just stare at screens.

Three months later, workshop efficiency went up and quality did not drop. The factory director summed it up in one sentence: for processes that require a person to nod, put the person in the middle of the path. For processes that only require a person to watch, put the person up high, but don't forget to have them come down and walk around regularly.

Later, whenever each new production line was launched, they first spent a week determining which nodes needed stamping and which nodes only needed watching. The old workers called this "define the posts first, then assign the people."

Human-in-the-Loop (HITL)

Humans are inside the decision loop; every critical operation in the system requires human approval or confirmation before it can proceed. Suitable for high-risk compliance, security, and financial decision scenarios.

Human on the Loop (HOTL)

Humans are at the oversight layer; the system runs autonomously, but humans continuously monitor and intervene when there is an anomaly or when escalation is needed. Suitable for batch processing, high real-time requirements, and controllable-risk scenarios.

Story Mapping

  • Approval line: HITL, each process step requires a human sign-off before release.
  • Monitoring line: HOTL, runs automatically, humans intervene only on exceptions.
  • Approval-line throughput is only 40% of the monitoring line: the throughput cost of HITL, where humans become the speed bottleneck.
  • A two-degree temperature shift went undetected: a monitoring blind spot in HOTL, relying on alarms but lacking proactive spot checks.
  • Hourly on-site spot checks: HOTL's supplementary mechanism — you can't rely on automated alerts alone; you need active patrols.

In the project

At every key decision point in designing an AI system, first mark the risk level. High-risk nodes (involving money, compliance, or personal safety) default to HITL, with humans in the decision path, and the system must not execute directly past the human. Medium- and low-risk nodes (batch classification, content summarization, routine replies) can use HOTL, but must be equipped with three things: clear escalation thresholds, an active sampling inspection mechanism, and a design that allows humans to take over manually at any time. The worst design is nominally HOTL but without escalation and inspection—that is in fact human-out-of-the-loop.

京ICP备2026014272号