← Back to concept explainers

Concept Fable

Decoder-only

Conclusion first:Decoder-Only is an architecture that uses only a stack of Transformer decoders, and most modern GPT-style large language models follow this route. It places the input prompt, context, examples and already-generated content all in the same sequence, predicting the next token autoregressively. In enterprise Agents, this architecture suits general generation and tool calling, but depends heavily on context engineering, retrieval filtering, prompt boundaries and cost control.

Decoder-Only · Architecture

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The Merchants' Guild with Only a Reply Room

The North Bridge Chamber of Commerce once had three buildings: the Letter Reading Building, the Account Verification Building and the Reply Building. Later they discovered that most clients did not want standalone reports, but directly executable replies, checklists, code and next-step actions.

The new president decided to change the system into a larger reply room. All materials, questions, rules and historical records were laid out on the same long table, and the secretary read from left to right, then continued writing further along.

At first, this looked too crude. Without a dedicated letter-reading house, how could the system understand complex materials? The chairman's remedy was to keep adding preamble explanations, and as a result the long table grew longer and longer, and the clerk was exhausted before even starting to write.

The second fix was to stuff everything in. The client was very satisfied that "everything was given," but the bookkeeper found that the cost of each letter rose, and old materials could interfere with new tasks.

The real fix is not to restore the three buildings, but to learn to put tasks, materials, tool results and outputs into the same sequence and organize them by causal order.

In this way, the reply room could both read and write, and could also imitate formats from examples. It is not necessarily the only structure for understanding tasks, but it became one of the simplest and strongest routes to large-scale general generation.

The chamber of commerce paid a price too: context management, retrieval selection, system prompts, tool returns and output budgets all had to be stricter, otherwise everything would end up at the same expensive long table.

The chairman finally said the building's advantage was unity, and its risk was also unity. Once everything became a continuation problem, whoever decided what went onto the long table determined the system's ceiling. Later the chamber of commerce also learned restraint. Not every sheet should be laid on the long table; old accounts, examples and rules had to be selected by task. Nor should every reply be continued indefinitely; when it was time to check the cabinet, ask a person, or stop writing, the results on the long table had to be handed over. The reply room thus resembled both a reading place and a writing place, but its order always came from the continuous strip of paper. Later the clerks also set cleaning rules for the long table. Expired sheets were removed promptly, and key instructions were placed at the front, lest the unified room become a junk room that swallowed anything. What you put in often mattered more than the room itself. With the long table in order, continuation was stable. With clear trade-offs, it was not slowed by old paper.

Reveal

This story is about: decoder-only

Decoder-Only is an architecture that uses only a stack of Transformer decoders, and most modern GPT-style large language models follow this route. It places the input prompt, context, examples and already-generated content all in the same sequence, predicting the next token autoregressively. In enterprise Agents, this architecture suits general generation and tool calling, but depends heavily on context engineering, retrieval filtering, prompt boundaries and cost control.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Only the reply room: Decoder-Only architecture
  • Long table: a unified prompt/context sequence
  • Constantly adding preamble: context-stacking patches
  • Cramming everything in: high-cost, low-selectivity context use
  • Causal order: generation constraints of Decoder-Only
  • Unified reading and writing: use the same model for understanding and generation
  • Final insight: general generative capability turns context governance into a core engineering problem

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

causal attentionautoregressiveGPTnext-token prediction
京ICP备2026014272号