Vector DB — A purpose-built search warehouse for AI

Teams often start out thinking “if we create embeddings and put them in a vector DB, search will automatically get better,” only to find the results get weirder. A vector DB isn’t a magic search engine—it’s a specialized warehouse designed to quickly find “things with similar meaning.” Skip this premise, and you’ll head in the wrong direction from the start.

---

Background / why this term now

To make a chatbot “answer based on our company’s documents,” most people adopt a RAG (retrieval-augmented generation) architecture. It first finds relevant documents for a given question, then passes them to the model. A vector DB is what handles “how to find those relevant documents.” IBM defines a vector DB as “a database that stores data as numerical representations called vector embeddings and searches based on semantic similarity rather than exact keyword matches.”

As RAG spreads rapidly across internal chatbots, customer support, and code search, vector DBs are often treated as a component you “just bolt on.” But cases are also increasing where teams adopt one without first clarifying why it’s needed—or when it isn’t.

Key data & current landscape

According to a December 2025 report from market research firm MarketsandMarkets, the global vector DB market is projected to grow from about $2.65B in 2025 to about $8.95B in 2030, a 27.5% CAGR. The 2025 figure is also more than 30% higher than 2024 (about $2.0B), tracking the broader adoption of LLMs and RAG.

The product landscape splits into three broad camps.

구분	대표 제품	특징
관리형 전용 벡터DB	Pinecone 등	운영 부담 없이 사용량 기반 과금
오픈소스 전용 벡터DB	Milvus, Weaviate, Qdrant, Chroma 등	직접 운영, 대규모 확장에 유리
기존 DB의 벡터 확장	PostgreSQL(pgvector), Redis, Elasticsearch, MongoDB Atlas 등	이미 쓰는 인프라에 검색 기능만 추가

In Korea, examples are emerging as well. Samsung SDS reported that after applying RAG and vector search to cloud tech support inquiries, about 68% of users resolved issues on their own. However, this is an internally measured figure tailored to a specific service, and it’s not guaranteed to transfer directly to other organizations.

Deep dive

1) What exactly is a vector DB?

When you feed data like text or images into an embedding model, you get numerical coordinates (vectors). The more similar the meaning, the closer these coordinates lie. A vector DB stores these coordinates at scale and, when a new query arrives, quickly finds “the nearest coordinates.” Comparing every vector in a set of millions to billions each time would be slow, so it uses approximate nearest neighbor (ANN) algorithms like HNSW (Hierarchical Navigable Small World) to gain speed. According to Pinecone’s explanation, HNSW builds a hierarchical graph, scanning broadly in upper layers and narrowing precisely in lower layers—one of the representative indexing techniques for vector similarity search.

2) Common misconceptions—and why they arise

The most common misconception is that “vector search always returns the correct answer.” As the name ANN—Approximate Nearest Neighbor—suggests, you trade some accuracy (recall) for speed. Another is thinking “hooking up a vector DB completes RAG.” In reality, how you chunk documents, which embedding model you use, and how you re-rank results often have a bigger impact on quality. A vector DB is just one part of that pipeline.

3) Practical decision criteria

Evaluate four things first when deciding whether to adopt one. First, the number of vectors to store (data scale). For tens of thousands to hundreds of thousands, you may not need a separate system at all. Second, the required search latency—do you need real-time responses, or is batch acceptable? Third, whether hybrid search is needed alongside keyword search (proper nouns, code, numbers). Fourth, whether you already operate a database. The cost of adding a new system is always a real cost.

4) When it’s useful—and when it’s pointless

Vector DBs shine when you need semantic search: internal document chatbots, retrieving similar past customer inquiries, product recommendations, image similarity search. Conversely, for lookups that require exact matches like order numbers or member IDs, or when the dataset is so small that brute-force comparison is instantaneous, there’s little reason to run a separate vector DB. In such cases, adoption often only adds complexity.

In practice — 5 rules for reading vector DBs correctly

1. If you have tens of thousands of items or fewer, first consider adding the pgvector extension to the PostgreSQL you already use instead of introducing a new system.
2. “Vector search = answer search” is a fallacy. ANN is approximate, so decide upfront whether you prioritize recall or speed.
3. If proper nouns, code, or numbers are frequently queried, pure semantic search won’t suffice. Consider a hybrid setup that includes keyword search.
4. Changing the embedding model means you must regenerate all previously stored vectors. Pre-calculate the cost of model swaps to stay safe.
5. If you lack dedicated ops staff, start with a managed service, then move to open source when data size makes costs burdensome. This sequence reduces the cost of failure.