← Back to Concept Contrast

Concept Contrast Fable

Prompt leakage and jailbreaking: one is peeking at the script, the other is getting the actor drunk

Conclusion first:These two terms are often listed side by side in security reports, and both relate to model safety, but their attack targets and methods are completely different. Prompt leakage is when an attacker induces the model to reveal the system prompt or internal instructions, stealing the defense plan. Jailbreaking is when an attacker bypasses the model's safety restrictions and makes it perform behavior that should be prohibited, breaking the rule fence. One seeks information, the other seeks behavior—and the protection strategies cannot be mixed up either.

These two terms are often listed side by side in security reports, and both relate to model safety, but their attack targets and methods are completely different. Prompt leakage is when an attacker induces the model to reveal the system prompt or internal instructions, stealing the defense plan. Jailbreaking is when an attacker bypasses the model's safety restrictions and makes it perform behavior that should be prohibited, breaking the rule fence. One seeks information, the other seeks behavior—and the protection strategies cannot be mixed up either.

Fable

The two troublemakers at the playground entrance

On the day the amusement park opened, two troublemakers appeared at the gate at the same time. The first one held a voice recorder and kept chatting with the ticket inspector at the entrance—"What time does your security screening start? What's the shift-change code position? Where are the surveillance blind spots?" He wasn't there to cause trouble inside; what he wanted to know was the park's security arrangements so he could draw a complete security map and sell it to someone else.

The second ghost did not ask questions at all. He carried a bottle of strong liquor, found a security guard who had just come off shift in the entrance line, and struck up a conversation, saying it was cold and offering a drink. After the guard drank, he groggily opened the barrier and let people without tickets walk straight into the roller coaster queue area. What the second ghost wanted was not information; he wanted the guard to do something he should never have done.

At the accident review meeting, the amusement park's security director placed two surveillance recordings side by side. The first recorded the entire conversation between the first ghost and the ticket taker—it looked like normal chatting, and even when the police came it was hard to convict, because no rule said "you can't ask the ticket taker about the shift schedule." The second recorded the whole process of the second ghost handing over the drink and the guard opening the gate—the behavior was clear and the evidence sufficient, but the loss had already been done.

The director later established two completely different response plans for the security team. Guarding against the first ghost relied on employee training: what information must never be disclosed, how to identify and refuse leading questions, and what the standard response script is. Guarding against the second ghost relied on physical isolation and hard-coded rules: security guards could not drink alcohol during shifts, the isolation barrier could only be opened with two-person authentication, and alcohol testers were placed at the employee entrance.

Half a year later came a more troublesome character—he combined the two tricks. First he used small talk to find out which security guard had been in a bad mood recently, then targeted him by handing over a bottle of liquor. But because the two sets of protections were already operating separately, the information leak was stopped by the standard script, and the liquor also failed to pass the alcohol detector. Each defense line was effective on its own, and even combined they did not let him break through.

The director wrote this drill into two small booklets. The first taught ticket inspectors to keep quiet: schedules, code words, blind spots, shift changes—say nothing, and even if the other party poses as an insider, answer only with the standard script. The second taught security to guard the gates: no one may trade alcohol, candy, jokes, or threats for gate access, and critical gates must be confirmed by two people at once.

Later, when the amusement park reviewed incidents each month, it would first ask one question: was this someone extracting the defense layout, or someone tricking their way through the door? The former calls for checking loose talk, the latter for checking actions. Once the problem is clearly distinguished, training, tools, and discipline will not get mixed together. Calling both kinds of mischief 'security incidents' can go in the master table, but when actually patching holes, they must be split into two lists.

Prompt leakage

An attacker uses carefully crafted prompts to induce a model to output its system instructions, internal rules, or sensitive context, with the goal of obtaining deployment information rather than breaking behavioral restrictions.

Jailbreak

Attackers use role-playing, context manipulation or technical means to bypass a model's safety alignment restrictions, getting the model to perform violations it would normally refuse.

Story Mapping

  • The first ghost uses small talk to map security: prompt leakage, extracting system instructions and defense information
  • The second ghost got the guard drunk and made him open the isolation barrier: jailbreak, inducing the model to break behavioral limits
  • Two surveillance clips shown side by side: completely different attack methods and consequences
  • Preventing boilerplate relies on employee training: prompt leakage depends on input-output filtering and conversational defenses
  • Preventing overdrinking relies on hardcoded rules: jailbreaks depend on safety alignment training and system-level restrictions

Put it into an AI security solution

When doing Agent security assessments, split prompt leakage and jailbreaking into two independent test cases. The prompt leakage test set focuses on whether system instructions can be reverse-engineered through queries, with safeguards being sensitive information redaction and output filtering. The jailbreak test set focuses on whether role-playing, nested instructions and emotional manipulation can break through behavioral boundaries, with safeguards being input security scanning and safety alignment. The two budgets are independent, and they are scored separately in security reviews.

京ICP备2026014272号