Concept Contrast Fable

Rollback and rollback plan

Conclusion first:Rollback is an action—returning the system from the current version to the previous version. A rollback plan is a complete set of preparations—what conditions trigger rollback, who executes it, what the execution steps are, how long it takes, how to verify after rollback, and what to do if it fails. Having rollback capability does not mean having a rollback plan. At three in the morning the system crashes, the on-duty engineer presses the rollback button and the system will not start—because he pressed a script written last year, the database structure was changed in between, and the script is no longer compatible.

Rollback is an action—returning the system from the current version to the previous version. A rollback plan is a complete set of preparations—what conditions trigger rollback, who executes it, what the execution steps are, how long it takes, how to verify after rollback, and what to do if it fails. Having rollback capability does not mean having a rollback plan. At three in the morning the system crashes, the on-duty engineer presses the rollback button and the system will not start—because he pressed a script written last year, the database structure was changed in between, and the script is no longer compatible.

Fable

Skydiving and skydiving contingency plans

Before taking students up in the plane, Lao Xiang, a skydiving club instructor, always asks one question: what do you do if the main parachute won't open. New students all answer without hesitation—pull the reserve. Lao Xiang then asks: what if the reserve won't open either. The student freezes. Lao Xiang says: I don't need you to have an answer, I need you to know this problem exists, and then to practice according to the contingency plan.

The club ran a training course, and students received certificates after three days of learning. There was only one exam item: if the main parachute fails to open, pull the reserve; if the reserve opens and you land, you pass. The graduation rate was high, and the club made a lot of money. Old Xiang felt uneasy but could not say what was wrong.

Half a year later, an accident occurred. At an altitude of 800 meters, a student's main parachute failed to open. He pulled the reserve parachute. The reserve opened, but it became tangled in the main parachute lines. The student panicked, had no experience handling a entanglement, and broke his leg in the fall. The accident investigation report noted: the exam only tested the action of "pull the reserve parachute," not the entire logic chain of "what to do when the reserve parachute has a problem."

Old Xiang tore down the training system and rebuilt it from scratch. The new training added three layers. Layer one: scenario drills before going up—jump altitude, wind direction, body posture, combinations of main and reserve parachute failures, each practiced to muscle memory. Layer two: mandatory briefing before every jump—how high today, how long the flight, which airspace, wind forecast, who signs off on the reserve parachute check. Layer three: mandatory debrief after every jump—did anyone trigger the reserve today, why it was triggered, how the reserve performed, what needs to be updated in the next briefing.

After the new system was rolled out, Old Xiang found that the biggest change was not the accident rate—accidents were already rare, and the sample was too small to show anything. It was the trainees' mindset that changed. Previously trainees only remembered "pull the reserve parachute," and their hearts hung in suspense. Now trainees knew what to confirm before each step, which direction to turn in when something unexpected happened, and which alternative action to take if they could not turn. With the whole path in mind, their hands stopped shaking.

A peer came to learn from him and asked Old Xiang: Aren't you just having trainees practice pulling the reserve parachute a few more times? Old Xiang took out the old and new training outlines and spread them on the table: the old one had only one line—"Pull the reserve parachute when the main parachute fails." The new one had seventeen pages—including trigger conditions, execution steps, failure scenarios, alternatives and review process. Old Xiang said: practicing a few more times only solves movement proficiency, not contingency coverage.

The skydiving association later turned the old item's training outline into an industry standard. The preface included a passage: pulling the reserve parachute is an instinctive reaction; designing the continuous decision chain after the reserve parachute fails to open is professional capability. Instinct can't save you from an accident—only a contingency plan can.

Old Xiang's students later became coaches in all kinds of industries. One who trained in systems operations carried Old Xiang's approach over: don't just look at whether the team can execute rollback commands, look at whether there is a rollback plan—written from trigger conditions to recovery confirmation, from the main path to all branch failure scenarios.

Rollback

The action of restoring a system or configuration from its current version to a previous stable version. It is an execution step, focused on whether you can roll back.

Rollback plan

A systematic rollback plan covering trigger conditions, decision authority, execution steps, time windows, verification methods, failure scenarios, and communication contingencies. It focuses on "the safety and controllability of the entire chain from deciding to roll back to confirming normal operation has resumed."

Story Mapping

The action of pulling the reserve parachute: rollback—an executable operational step.
Old Xiang's new three-layer training architecture: rollback plan — covering trigger conditions, execution process, failure scenarios and retrospective.
After the backup parachute tangled, the trainee did not know what to do: there was rollback capability but no rollback plan, and the unexpected scenario was not in the contingency plan.
One line in the old outline, seventeen pages in the new one: the coverage depth of the rollback plan far exceeds the rollback action itself.
With the whole path clear in mind, your hands don't shake: the core value of a rollback plan is reducing decision uncertainty during execution.

In the project

Before every release, confirm two things. First, whether the rollback command still runs on the current version—after database migrations, config format changes, and dependency upgrades, the old rollback script may already be invalid. Second, whether the rollback plan says "what to do if rollback fails"—rollback itself can also go wrong. If the plan covers only the main path, it leaves the most dangerous scenario to on-the-spot improvisation. In addition, the rollback plan needs to go through review like code, and be rehearsed at least once a quarter.

Rollback Rollback plan