← Back to concept explainers

Concept Fable

Vision Transformer

Conclusion first:Vision Transformer (ViT) cuts images into patches and uses a Transformer to model the relationships between image blocks as if processing a token sequence. It introduces self-attention into vision tasks and is suitable as a foundation for visual understanding such as classification, retrieval, and detection. In enterprise multimodal Agents, the ViT approach helps systems process screenshots, receipts, quality inspection images, and interface visual information, but the number of patches significantly affects computation and latency.

Vision Transformer, ViT · Architecture

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The Appraisal Institute That Cut Paintings into a Chessboard

The painting academy wanted apprentices to read large paintings. Traditional masters first looked at lines and corners, pushing from local texture all the way to the meaning of the whole. This worked well for small flower-and-bird paintings, but with long crowd scrolls, they often managed the feathers but not the whole. The director initially worried that cutting the painting would destroy its mood. A new painter spread out a long scroll: if you stare only at local brushstrokes, the relationship between distant military flags and nearby fleeing crowds becomes invisible.

The new painter cut the long scroll into equal-sized small squares, treating each as a word token. The apprentice first learned roughly what each square contained, then had all the small squares cross-reference each other: this flag connects to that city gate, the distant dust echoes the nearby galloping horse. The first cut had no position markers, so the apprentice took a riverbank fragment for a mountain path and connected the distant city gate behind the nearby horse hooves—the whole painting's space fell into disorder.

At first the slices were too coarse and facial expressions were lost; too fine, and the table was covered in scraps that were hard to handle. The painter experimented repeatedly, found a suitable slice size, and labeled each slice with its original position so a small piece would not forget where it belonged in the painting. The slice-size experiments also had their setbacks. Too large a slice was like a whole long sentence, hard to compare with others; too small a slice lost the object's outline and left only meaningless color blocks.

After several rounds, the apprentice no longer only looked locally along the brushstrokes, but could read the entire painting as a string of interrelated fragments.

Later, when the apprentice read a battle map, he first looked at each piece, then had the flags, cavalry, city gates, and dust cross-reference one another. Distant small pieces influenced near judgments, and near actions also rewrote distant meanings.

The painting academy also did not discard the old masters' local eye for detail. Fine lines, corners, and brushstrokes still mattered, but long scrolls required seeing distant relationships earlier. Local craft and global comparison together made reading the image reliable.

Later the painting academy used slice-based image reading for new paintings and also for restoring old scrolls. When a corner was missing, apprentices could infer content from surrounding fragments and distant clues; but if the original painting was too damaged, they still had to admit they could not see clearly. Mutual attention can supplement clues, but cannot create truth out of nothing.

There is more than one way to read an image. Cutting the painting into pieces and letting the pieces attend to one another brings the distant and near relationships in a long scroll into judgment earlier.

Reveal

This story is about: Vision Transformer

Vision Transformer (ViT) cuts images into patches and uses a Transformer to model the relationships between image blocks as if processing a token sequence. It introduces self-attention into vision tasks and is suitable as a foundation for visual understanding such as classification, retrieval, and detection. In enterprise multimodal Agents, the ViT approach helps systems process screenshots, receipts, quality inspection images, and interface visual information, but the number of patches significantly affects computation and latency.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Appraisal office: vision model system
  • The Painting Is Cut into a Chessboard: Image Patches
  • Each region reports separately: local processing lacks global relationships
  • The hall that sees relationships: Transformer self-attention
  • Tile coordinates: positional encoding
  • Quality inspection, OCR, image-text understanding: enterprise multimodal applications
  • Final insight: visual input can also be serialized, but the cost rises with the number of image patches

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

image encodertransformerVLMpatch embedding
京ICP备2026014272号