← Back to concept explainers

Concept Fable

Memory compression

Conclusion first:Memory Compression compresses long conversations, historical observations, retrieved materials or execution traces into shorter summaries, structured facts or retrievable memory, so they can fit into the subsequent context budget while preserving as much task-useful signal as possible. Enterprise agents need it to control token costs, reduce context noise, and keep long-running tasks from losing key state across multiple rounds of execution.

Memory Compression · Agent memory

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The archive compressed ten carts of old cases into a single travel ledger

The Jiangwan Archives kept case files for the city's inspection team. In the early years there was little business, and each time the inspectors went out they loaded all related past files onto the cart, flipping through them slowly on the road, always able to find a useful clue.

Later, as cases multiplied, the files piled up to ten carts. Before the inspector even reached the site, the horse was already exhausted; to save trouble, the clerk brought only the three most recent volumes, and as a result key names from old cases were often missed.

The curator first had the scribes write summaries. Each file was compressed into three lines, but three lines were too short, leaving only conclusions without evidence, timing, or exceptions. Inspections acted on the summaries and several times lumped similar cases into the same category.

Another scribe did the opposite, keeping all details and deleting only pleasantries and filler. The cart became a little lighter, but every trip still required many volumes, and the truly useful clues were still buried in the pile of paper.

An experienced case master began organizing by layers. He compressed the long scroll into case history, reliable facts, unresolved questions, character relationships, and next-action prompts; places that could not be determined were clearly marked as pending investigation rather than pretending to already understand.

He also ruled that only facts that repeatedly influenced judgment would enter the commonly used travel ledger. One-off noise stayed in the original scroll, to be looked up again when needed.

At first the inspector worried that compression would lose things. The old case master had them compare the compressed ledger with the original file after each case: which summaries helped, which omissions caused misjudgment, and which old facts had expired.

A few months later, the inspectors went out carrying only a travel log and a small index of original volumes. The team moved faster and less often treated old cases as new ones. The curator finally understood that the value of memory is not in moving the past unchanged into today, but in preserving the signals that future decisions truly need. Later the travel log was revised once a month: what proved useful stayed, what had misled was rewritten, and what had become invalid was marked in red. Compression is not making old matters shorter, but letting old matters continue to serve new judgments. The inspectors also learned not to blindly trust the travel log. When facing a judgment of great consequence, they would follow the index back to the original volume to confirm that compression had not smoothed away key exceptions.

Reveal

This story is about: memory compression

Memory Compression compresses long conversations, historical observations, retrieved materials or execution traces into shorter summaries, structured facts or retrievable memory, so they can fit into the subsequent context budget while preserving as much task-useful signal as possible. Enterprise agents need it to control token costs, reduce context noise, and keep long-running tasks from losing key state across multiple rounds of execution.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Archive: a system that preserves an Agent's historical context and task memory
  • Ten carts of old cases: long conversations, logs, retrieved snippets and execution traces
  • Three-line summary: over-compression, losing verifiable details
  • Layered Organization: Structured Memory Such as Summary, Facts, Open Questions, Entities, Next Actions
  • Common travel log: high-value signals retained in subsequent prompts or memory stores
  • Check the original files: when necessary, return to original records or external knowledge bases to supplement evidence
  • Final insight: memory compression isn't just about getting shorter—it's about preserving the information needed for future action within a limited context

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

context windowsummarizationlong-term memorysemantic memorymemory consolidationtoken budget
京ICP备2026014272号