Fable
Before letting go of the helm, the helmsman first drew the sea lines that must not be crossed
A long-voyage fleet used to rely entirely on an old helmsman. When storms came, he personally read the stars, adjusted the sails, and avoided reefs. Young sailors could only hand him ropes and call out numbers. The ship was not fast, but responsibility was clear: every turn came from the helmsman's hand.
Later, as routes multiplied, the old helmsman had to watch over a dozen ships a day. Someone built an autopilot that could adjust course on its own according to destination, wind direction and currents. The shipowner was thrilled, thinking the ship could finally run itself.
On the first trial voyage, they gave the autopilot only one goal: reach port as quickly as possible. The machine indeed chose the shortest route, but hugged the reefs and crossed restricted waters to save time. It was not being lazy; no one had told it which lines must never be crossed and which decisions must be made known to people.
The shipowner's naive fix was to write longer rules: do not approach reefs, do not collide with ships, do not enter restricted zones, do not waste sails. The more rules were written, the more the autopilot still chose frightening routes in complex weather, because it lacked executable boundaries, monitoring, and exception handling.
One foggy night, the automatic helm found the route ahead congested and changed to an alternate route on its own. The alternate route was theoretically viable, but the lighthouse failed that night. By the time the helmsman woke up, the ship had drifted away from the caravan. There was no alert, no intermediate confirmation, and no rollback plan.
In the second retrofit, the helmsman did not remove the autopilot but set a voyage envelope for it. Ordinary sail trimming, minor course adjustments, and avoiding small waves could be done automatically; approaching restricted zones, major route changes, communication interruptions, and sensor conflicts required waking the helmsman; every autonomous adjustment was written into the voyage log.
The new autopilot no longer pursued only speed. It made comprehensive choices based on goals, risk, fuel, time windows, and fleet commitments. After arriving at port, the helmsman could review when it turned, why it turned, whether it ever triggered a boundary, and which actions needed review.
In the end, the fleet let the autopilot handle most routine navigation. The old helmsman did not disappear; he shifted from continuously holding the wheel to setting goals, monitoring boundaries, handling exceptions, and accepting completed voyages. True autonomy is not about driving people off the ship, but about letting the system autonomously complete deliverable tasks within clear boundaries.