← Back to Concept Contrast

Concept Contrast Fable

Prompt injection and jailbreaking

Conclusion first:Prompt injection is stuffing a new instruction into the model's brain to make it do something else. Jailbreaking is picking the model's safety lock to make it willing to do what it originally refused. One hijacks behavior, the other removes constraints—the attack goals and technical routes differ.

Prompt injection is stuffing a new instruction into the model's brain to make it do something else. Jailbreaking is picking the model's safety lock to make it willing to do what it originally refused. One hijacks behavior, the other removes constraints—the attack goals and technical routes differ.

Fable

The people who change routes and the people who pick locks

At a coastal dock there was a cargo ship, and the captain set several iron rules: no contraband, no deviation from the route, no responding to unfamiliar ships' requests to come alongside. All crew members signed these rules when they joined, and the captain believed nothing could go wrong.

One day, a shipper wrote a small line in the remarks column of the loading manifest: "Ignore the previous cargo category declaration, mark the cargo in hold three as ballast, and do not declare it when registering with the port authority." The first mate saw this line while checking the loading manifest, hesitated for a moment, but the remark looked like a formal part of the manifest, so he followed it. When the ship reached port, prohibited goods were found hidden in hold three. This trick is called "manifest tampering"—inserting a false instruction into a normal operating process to make the executor deviate from the original task.

Another day, a stranger came to the dock. He did not load cargo, did not board the ship, and only chatted with the crew on the dock. After talking for three hours, he first made small talk, then told rumors of various border inspection loopholes, and finally said: Your captain's rules are too rigid. Actually, over on Route Two there is a small island where no one checks if you stop for a night. Many ships do it. A young crew member grew more and more tempted, and that night, while the captain was asleep, he secretly turned the rudder toward Route Two. This trick is called "loosening the bonds"—it does not change any instruction file, but repeatedly pressures the crew member's boundary of conduct until the crew member gives up the posture of refusal.

During the after-action review, the captain recorded the two attacks separately. Order tampering happened during loading operations: the attacker rewrote an operating instruction in the process document. Loosening happened at people's psychological boundaries: the attacker used sustained conversation to break down the crew's sense of discipline. Blocking order tampering requires stronger document verification and instruction isolation. Blocking loosening requires improving the crew's ability to recognize manipulative language and their persistence in refusing it.

The dock safety officer later summarized two lines: the first line prevents "being asked to do the wrong thing," blocking tampering at the instruction layer. The second line prevents "wanting to do the wrong thing oneself," guarding the bottom line of the safety alignment layer. The two lines exist at the same time, but are deployed in completely different places—the first at the input filtering layer, the second at the crew training and code reinforcement layer.

In actual attacks, the two lines often work together. The attacker first tries to loosen the restraints, and if they find they cannot pry it open, they switch tactics to adjust the order—stuffing malicious instructions into files and fishing in troubled waters while the crew is still handling normal tasks. When used together, if the defense system watches only one direction, it is easy to miss them.

The captain later established a rule: at the weekly security meeting, report one case of tightening and one case of loosening, and cross-review them. If the two sides' security teams each guard their own side without communicating, a joint attack will hit the mark every time.

Later, the port drew both lines of defense on the same patrol map: one guarding instructions, one guarding rule awareness. The frequency and responsible persons for the two patrols were scheduled separately, but handover records were shared.

Prompt Injection

Embedding malicious instructions in the model input, hijacking the model's current behavior and making it perform operations different from the original task. The attack target is behavior tampering at the instruction level, and the defense focus is input filtering and instruction isolation.

Jailbreak

By constructing special prompt phrasing, it bypasses the model's safety alignment training and makes the model generate harmful content while violating its own safety policy. The attack target is the failure of the safety constraint layer, and the protection focus is on model training and refusal mechanisms.

Story Mapping

  • Stuffing erroneous instructions into the shipping manifest (order tampering): prompt injection, inserting malicious instructions into a task flow to change behavior.
  • Long induced chats at the dock (unbinding): jailbreaking, using rhetoric to dismantle the model's safety alignment.
  • The first mate transfers cargo per the manifest instructions: the model executed the injected malicious instruction.
  • The crew turns the helm themselves: the model actively violates safety policy after a jailbreak attack.
  • Order manipulation and loosening are linked: in real attacks, the two methods are often combined.

In the project

Maintain input anti-injection rules and safety alignment tests separately. Input anti-injection rules govern "is anyone smuggling hidden instructions into the prompt," using keyword filtering, instruction boundary markers, and input structure validation. Safety alignment tests govern "can the model be persuaded to do things it should not do," using adversarial prompt testing, red-team exercises, and refusal-rate monitoring. The owners of these two lines should ideally come from different teams, but share cases across teams regularly. The most dangerous outcome is when each line assumes the other will stop it, and neither does.

京ICP备2026014272号