← Back to concept explainers

Concept Fable

Milvus

Conclusion first:Milvus is an open-source distributed vector database designed for large-scale vector similarity search. It uses a storage-compute separation architecture and supports multiple index types (HNSW, IVF_FLAT, IVF_PQ, DiskANN, etc.), horizontal scaling, data persistence, and hybrid queries. Milvus is divided into an access layer (proxy), query nodes, data nodes, index nodes, and a storage layer (object storage + message queue), and each component can scale independently. In enterprise RAG, Milvus is suitable for scenarios with vector volumes from tens of millions to billions: high-concurrency online queries, continuous writes of incremental data, coexistence of multiple index strategies, and strict requirements for failure recovery and data persistence. It provides Python/Java/Go SDKs and a RESTful API, and integrates well with frameworks such as LangChain and LlamaIndex.

Milvus · Open-source vector database

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The weaving workshop owner learned to divide the yarn rather than add more

The largest weaving workshop in Jiangnan had several hundred looms running at the same time, and each loom needed to find the best-matching yarn from the warehouse. The old manager's method was to hang all yarn samples on a long wall, and workers ran over to compare them one by one.

As business grew, yarn varieties went from dozens to tens of thousands. That wall extended into more than a dozen rooms, and workers ran from morning to night, their legs more tired than their eyes. Experienced masters relied on memory, but new workers always confused similar yarns.

The owner first bought three larger walls. With larger walls, comparison did not get faster; instead it got slower because the paths traveled were longer. Three stewards separately managed the left, middle and right walls, and synchronization relied on shouting, producing more errors and omissions.

The engineer came by, took one look and said you need to split the yarn into several groups placed separately, each group just the right size to fit on a shelf within one person's reach. When querying, first determine which group the yarn belongs to, then search only within that group.

The owner worried that many rooms would lead to management chaos. The engineer gave him a master diagram: although all the shelves were scattered, they were managed centrally by one dispatch console. The console knew who was where, which shelf to check, and where new yarn should go once it arrived.

The first improvement was to give different racks different search methods: some use nearest-neighbor graphs to quickly find approximate matches, some group by buckets for large-scale coarse search, and others retain full-precision comparison for critical orders.

The second improvement was making the shelves expandable horizontally. During peak season, new warehouses connected to the central dispatch desk, and old queries did not need to be rebuilt from scratch. Workers still asked at the same window, but behind it work could be distributed to many rooms.

The weaving workshop no longer had to prop up its business with an endlessly extending wall. The owner understood that true capability lies in putting similarity search, grouping, indexing, scheduling, and scaling into a single warehouse order. Later, when the workshop received urgent orders again, the owner did not buy another long wall, but first asked which search method this batch of yarn should use, which group it belonged to, and whether fine comparison was needed. The central dispatch desk assigned the questions, and the results were steadier than the old master relying on memory. As scale grew, order mattered more than wall space. The owner came to understand that the larger the warehouse, the less it could rely on a single old master to remember the routes. Only when any worker could find similar yarn through the central dispatch desk could the business hold up.

Reveal

This story is about: Milvus

Milvus is an open-source distributed vector database designed for large-scale vector similarity search. It uses a storage-compute separation architecture and supports multiple index types (HNSW, IVF_FLAT, IVF_PQ, DiskANN, etc.), horizontal scaling, data persistence, and hybrid queries. Milvus is divided into an access layer (proxy), query nodes, data nodes, index nodes, and a storage layer (object storage + message queue), and each component can scale independently. In enterprise RAG, Milvus is suitable for scenarios with vector volumes from tens of millions to billions: high-concurrency online queries, continuous writes of incremental data, coexistence of multiple index strategies, and strict requirements for failure recovery and data persistence. It provides Python/Java/Go SDKs and a RESTful API, and integrates well with frameworks such as LangChain and LlamaIndex.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Yarn samples hanging on a long wall: single-machine in-memory vector retrieval, limited data volume and no persistence
  • Three walls, three managers each in charge of their own: simple multi-instance deployment, lacking unified scheduling and data consistency
  • Split into groups placed in different rooms: sharding, horizontally partitioning vector data
  • Unified management by the central scheduling console: the Milvus proxy layer, routing queries and writes to the corresponding nodes
  • Jump point graphs, bucketing, exact: multiple index types (HNSW, IVF, DiskANN, etc.)
  • Data persistence and incremental writes: a persistence mechanism based on object storage + message queue
  • Add more shelves rather than swapping in a bigger single unit: horizontal scaling, with query nodes and data nodes scaled independently
  • Splitting memory apart while keeping queries consistent: the core engineering challenge of distributed vector databases

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

Zillizvector indexANN searchembeddingsRAG
京ICP备2026014272号