What Is a Vector Database? HNSW, IVF, PQ, and ANN Search Explained

TL;DR — A vector database stores high-dimensional embeddings and answers nearest-neighbor queries using ANN indexes. StochasticSandbox: "HNSW is the default index in nearly every vector database — Qdrant, Weaviate, pgvector, Pinecone, Chroma all use it. PQ compresses vectors 8-32x. Start with pgvector if on Postgres <5M vectors, Qdrant/Weaviate for scale, Pinecone for managed, Chroma for prototyping." SystemsInternals: "At 10M vectors in 1536 dimensions, brute force is 15 billion operations. ANN reduces to tens of thousands with 95-99% recall. HNSW + PQ + re-ranking is what Pinecone, Qdrant, and Weaviate do under the hood." AbstractAlgorithms: "Flat: exact, small. HNSW: best default for RAG. IVF: large corpus. IVF-PQ: massive corpus, tight budget. Start with HNSW, move to IVF-PQ when scale forces it." BigDataBoutique: "HNSW reaches 95%+ recall, absorbs inserts without rebuilds. IVFFlat uses 2-5x less memory but recall drifts. HNSW is the default for production as of 2026." Youngju: "pgvector for Postgres <100M, Qdrant/Weaviate/Milvus for K8s 100M-1B, Pinecone for managed, Milvus GPU for >1B." Learn more with embeddings, embedding models, what is RAG, and cosine similarity.

SystemsInternals defines the core problem: "A vector database stores high-dimensional embeddings — floating-point arrays produced by transformer models, CLIP, or sentence-transformers — and answers one question faster than brute force: which stored vectors are closest to a query vector? At 10 million vectors in 1536 dimensions, comparing every pair is 15 billion operations. An approximate nearest-neighbor (ANN) index reduces that to tens of thousands with 95–99% recall."

StochasticSandbox frames the landscape: "Vector databases power every retrieval-augmented AI system. HNSW, IVF, and product quantization actually work, with tradeoffs across Pinecone, Weaviate, Qdrant, pgvector, and Chroma."

Vector Database Architecture

ANN Index Types Comparison

Vector Database Comparison

When to Choose Each Vector Database

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class IndexType(Enum):
    FLAT = "flat"
    HNSW = "hnsw"
    IVF = "ivf"
    IVF_PQ = "ivf_pq"

class VectorDatabase(Enum):
    PINECONE = "pinecone"
    WEAVIATE = "weaviate"
    QDRANT = "qdrant"
    MILVUS = "milvus"
    PGVECTOR = "pgvector"
    CHROMA = "chroma"

@dataclass
class VectorDatabaseSelector:
    """Select the right vector database and index for your use case."""

    def recommend(self, vector_count: int,
                  existing_postgres: bool = False,
                  need_hybrid_search: bool = False,
                  need_multi_vector: bool = False,
                  self_hosted: bool = True,
                  managed_ok: bool = False,
                  prototyping: bool = False) -> dict:
        """Recommend vector database based on requirements."""

        if prototyping:
            return self._result(
                VectorDatabase.CHROMA, IndexType.HNSW,
                "Zero config, pip install, perfect for prototyping")

        if existing_postgres and vector_count < 5_000_000:
            return self._result(
                VectorDatabase.PGVECTOR, IndexType.HNSW,
                "No new infrastructure, use existing Postgres")

        if existing_postgres and vector_count < 100_000_000:
            return self._result(
                VectorDatabase.PGVECTOR, IndexType.HNSW,
                "pgvector + pgvectorscale for "
                "StreamingDiskANN, 10-100x faster filtered search")

        if need_multi_vector or (need_hybrid_search and self_hosted):
            return self._result(
                VectorDatabase.QDRANT, IndexType.HNSW,
                "Named vectors + sparse vectors, "
                "payload index + filterable HNSW")

        if need_hybrid_search and not self_hosted:
            return self._result(
                VectorDatabase.WEAVIATE, IndexType.HNSW,
                "BM25 hybrid search, roaring bitmap filtering")

        if managed_ok and not self_hosted:
            return self._result(
                VectorDatabase.PINECONE, IndexType.HNSW,
                "Managed serverless, zero operational responsibility")

        if vector_count > 1_000_000_000:
            return self._result(
                VectorDatabase.MILVUS, IndexType.IVF_PQ,
                "Highest scale, GPU indexing, cloud-native")

        if self_hosted and 100_000_000 <= vector_count <= 1_000_000_000:
            return self._result(
                VectorDatabase.QDRANT, IndexType.HNSW,
                "Distributed cluster, high performance, self-hosted")

        return self._result(
            VectorDatabase.QDRANT, IndexType.HNSW,
            "Default: high performance, self-hosted, "
            "scalar + binary quantization")

    def recommend_index(self, vector_count: int,
                        memory_constrained: bool = False,
                        need_exact: bool = False,
                        active_writes: bool = True) -> dict:
        """Recommend ANN index type based on requirements."""

        if need_exact or vector_count < 100_000:
            return {
                "index": IndexType.FLAT,
                "reason": "Exact search, maximum recall, "
                          "perfect for small corpus or baseline",
            }

        if memory_constrained and vector_count > 100_000_000:
            return {
                "index": IndexType.IVF_PQ,
                "reason": "8-32x compression, "
                          "massive corpus with tight budget",
            }

        if not active_writes and vector_count > 10_000_000:
            return {
                "index": IndexType.IVF,
                "reason": "Lower memory, large mostly-static dataset",
            }

        return {
            "index": IndexType.HNSW,
            "reason": "95%+ recall, absorbs inserts without "
                      "rebuilds, best latency-recall tradeoff",
        }

    def _result(self, db: VectorDatabase,
                index: IndexType, reason: str) -> dict:
        return {
            "database": db.value,
            "index": index.value,
            "reason": reason,
        }

    def get_hnsw_params(self, m: int = 16,
                        ef_construction: int = 64,
                        ef_search: int = 40) -> dict:
        """Get HNSW parameters with recommendations.

        M: max connections per node (higher = more memory, better recall)
        ef_construction: build-time search depth (higher = better quality)
        ef_search: query-time search depth (higher = better recall)
        """
        return {
            "m": m,
            "ef_construction": ef_construction,
            "ef_search": ef_search,
            "recommendations": {
                "fast": {"m": 12, "ef_construction": 48, "ef_search": 32},
                "balanced": {"m": 16, "ef_construction": 64, "ef_search": 40},
                "high_recall": {"m": 32, "ef_construction": 128, "ef_search": 80},
            },
            "notes": (
                "HNSW absorbs inserts without quality loss. "
                "Graph adapts as nodes arrive. "
                "No rebuild step, no maintenance window, no recall drift. "
                "Uses 2-5x more memory than IVFFlat."
            ),
        }

    def get_ivf_params(self, nlist: int = 100,
                       nprobe: int = 10) -> dict:
        """Get IVF parameters with recommendations.

        nlist: number of clusters (higher = finer partitioning)
        nprobe: clusters to search at query time (higher = better recall)
        """
        return {
            "nlist": nlist,
            "nprobe": nprobe,
            "recommendations": {
                "fast": {"nlist": 256, "nprobe": 4},
                "balanced": {"nlist": 1024, "nprobe": 10},
                "high_recall": {"nlist": 4096, "nprobe": 32},
            },
            "notes": (
                "IVF partitions vectors into k-means clusters. "
                "Search restricts to nearest nprobe clusters. "
                "Lower memory than HNSW but recall drifts as data changes. "
                "Best for large, mostly-static datasets."
            ),
        }

    def get_pq_params(self, m: int = 8,
                      nbits: int = 8) -> dict:
        """Get product quantization parameters.

        m: number of subvectors (higher = better quality, more memory)
        nbits: bits per subvector code (256 centroids at 8 bits)
        """
        return {
            "m": m,
            "nbits": nbits,
            "compression": f"{2 ** nbits} centroids per subvector",
            "memory_reduction": "8-32x vs float32",
            "notes": (
                "PQ splits each d-dimensional vector into m subvectors, "
                "runs k-means on each, replaces with centroid ID. "
                "A 1536-d float32 vector (6KB) becomes 192 bytes: 32x. "
                "Most production deployments combine HNSW + PQ + re-ranking."
            ),
        }

Vector Database Checklist

• [ ] A vector database stores high-dimensional embeddings and answers nearest-neighbor queries
• [ ] At 10M vectors in 1536 dimensions, brute force is 15 billion operations
• [ ] ANN index reduces to tens of thousands of operations with 95-99% recall
• [ ] Every production RAG system, semantic search engine, and AI agent runs on this tradeoff
• [ ] HNSW is the default index type in nearly every vector database
• [ ] HNSW: graph-based, 95%+ recall, absorbs inserts without rebuilds
• [ ] HNSW: uses 2-5x more memory than IVFFlat
• [ ] HNSW: no rebuild step, no maintenance window, no recall drift
• [ ] HNSW: every modern vector database defaults to it (Qdrant, Weaviate, Pinecone, Milvus, pgvector)
• [ ] IVF: partitions vectors into k-means clusters, probes nearest clusters at query time
• [ ] IVF: lower memory than HNSW, but recall drifts as data changes
• [ ] IVF: best for large, mostly-static datasets where memory or build time dominates
• [ ] PQ: compresses vectors from 32-bit floats to compact codes, 8-32x memory reduction
• [ ] PQ: 1536-d float32 (6KB) becomes 192 bytes — 32x compression
• [ ] PQ: single most important technique for scaling past where raw vectors fit in memory
• [ ] PQ: typically costs 1-3% recall
• [ ] IVF-PQ: combines partitioning with compression for massive corpus and tight budget
• [ ] Flat: exact search, maximum recall, for baseline or small corpus
• [ ] Start with HNSW for most RAG systems, move to IVF-PQ when scale forces compression
• [ ] Validate with Flat, deploy with HNSW, scale with IVF-PQ
• [ ] Most production deployments combine HNSW + PQ + re-ranking pass (exact on top-100)
• [ ] Metadata filtering: pre-filtering, post-filtering, or in-graph filtering
• [ ] Qdrant: payload index + filterable HNSW, adaptive fallback for selective filters
• [ ] Weaviate: roaring bitmap intersection with inverted indexes on metadata
• [ ] Pinecone: payload index for metadata filtering
• [ ] pgvector: PostgreSQL WHERE clauses for filtering
• [ ] Pinecone: managed serverless, HNSW + IVF, PQ + binary quantization
• [ ] Weaviate: HNSW + composite, PQ (1.22+), BM25 hybrid, roaring bitmap filtering
• [ ] Qdrant: HNSW (custom), scalar + binary PQ, payload index + ranged filtering
• [ ] Milvus: HNSW, IVF, DiskANN, PQ + binary, field index, highest scale
• [ ] pgvector: IVF-Flat + HNSW, binary quantization + halfvec, PostgreSQL WHERE
• [ ] pgvectorscale: StreamingDiskANN, 10-100x faster filtered search than vanilla pgvector
• [ ] Chroma: HNSW, zero config, pip install, for prototyping only
• [ ] Choose pgvector if already on Postgres and fewer than 5M vectors
• [ ] Choose pgvector + pgvectorscale for Postgres with up to 100M vectors
• [ ] Choose Qdrant or Weaviate for production self-hosted with hybrid search
• [ ] Choose Qdrant for multi-vector (ColBERT) and sparse+dense hybrid
• [ ] Choose Pinecone for managed serverless with zero ops responsibility
• [ ] Choose Milvus for highest scale (>1B vectors) with GPU indexing
• [ ] Choose Chroma only for prototyping and hackathons
• [ ] HNSW parameters: M (connections), ef_construction (build depth), ef_search (query depth)
• [ ] Higher M and ef = more memory but better recall
• [ ] IVF parameters: nlist (clusters), nprobe (clusters to search)
• [ ] Higher nprobe = better recall but slower query
• [ ] PQ parameters: m (subvectors), nbits (bits per code)
• [ ] DiskANN: SSD-friendly graph index for billion-scale on single node
• [ ] Binary quantization + reorder saves 4x to 32x on disk and RAM
• [ ] Read embeddings for vector fundamentals
• [ ] Read embedding models for model selection
• [ ] Read what is RAG for RAG architecture
• [ ] Read cosine similarity for distance metrics
• [ ] Read hybrid search for BM25 + vector
• [ ] Read reranking for post-retrieval precision
• [ ] Test: HNSW achieves 95%+ recall with sub-10ms latency at 1M vectors
• [ ] Test: IVF-PQ achieves 90%+ recall with 32x memory reduction at 100M vectors
• [ ] Test: metadata filtering returns correct results with selective filters
• [ ] Test: recommendation logic selects correct database per use case
• [ ] Test: HNSW absorbs inserts without recall degradation
• [ ] Document index type, parameters, database choice, and scaling strategy

FAQ

What is a vector database and how does it work?

A vector database stores high-dimensional embeddings (floating-point arrays from transformer models) and answers nearest-neighbor queries using approximate nearest-neighbor (ANN) indexes. StochasticSandbox: "Vector databases power every retrieval-augmented AI system. HNSW, IVF, and product quantization actually work, with tradeoffs across Pinecone, Weaviate, Qdrant, pgvector, and Chroma. HNSW is the default index type in nearly every vector database. Qdrant, Weaviate, pgvector, Pinecone, and Chroma all use it as their primary or sole index structure." SystemsInternals: "A vector database stores high-dimensional embeddings and answers one question faster than brute force: which stored vectors are closest to a query vector? At 10 million vectors in 1536 dimensions, comparing every pair is 15 billion operations. An ANN index reduces that to tens of thousands with 95-99% recall." How it works: (1) Store vectors with metadata in a collection. (2) Build an ANN index (HNSW, IVF, or IVF-PQ). (3) At query time, embed the query and search the index. (4) Return top-k nearest vectors with metadata. (5) Apply metadata filters before, during, or after vector search.

What is HNSW and why is it the default vector index?

HNSW (Hierarchical Navigable Small World) is a graph-based ANN index that offers the best recall-latency tradeoff for collections under ~100M vectors. StochasticSandbox: "HNSW, published by Malkov and Yashunin in 2018, is the default index type in nearly every vector database. Qdrant, Weaviate, pgvector, Pinecone, and Chroma all use it as their primary or sole index structure. HNSW generally wins on query quality and latency. IVF wins on memory and disk-friendliness." BigDataBoutique: "HNSW is a graph-based vector index that reaches 95%+ recall out of the box and absorbs inserts without rebuilds, but uses 2-5x more memory than IVFFlat. The single most useful property for production: HNSW absorbs inserts without quality loss. The graph adapts as nodes arrive. No rebuild step, no maintenance window, no recall drift. That is why every modern vector database defaults to HNSW." AbstractAlgorithms: "Start with HNSW for most RAG systems, then move to IVF or IVF-PQ when scale and cost force compression."

What is the difference between HNSW, IVF, and product quantization?

HNSW is a graph-based index with the best recall-latency tradeoff, IVF partitions vectors into clusters for lower memory, and PQ compresses vectors 8-32x for massive scale. AbstractAlgorithms: "Flat: exact, simple, expensive at scale. HNSW: best default for online RAG when recall matters. IVF: scalable partitioned search with moderate complexity. IVF-PQ: compressed large-scale retrieval for memory-constrained systems. Start with HNSW, validate with Flat, move to IVF-PQ only when scale or cost requires it." SystemsInternals: "IVF partitions vectors into k clusters via k-means. Search restricts comparison to vectors in the nearest w clusters. Product quantization splits each d-dimensional vector into m subvectors, runs k-means on each, replaces with centroid ID — a single byte. A 1536-d float32 vector (6 KB) becomes 192 bytes: 32x compression. Most production deployments combine HNSW with PQ and a re-ranking pass." StochasticSandbox: "PQ compresses vectors from 32-bit floats to compact codes, reducing memory by 8-32x. It is the single most important technique for scaling vector search past where raw vectors fit in memory."

Which vector database should you choose?

Choose based on your scale, existing infrastructure, and operational preferences. StochasticSandbox: "For most teams building RAG: start with pgvector if already on Postgres and fewer than 5M vectors. Move to Qdrant or Weaviate when you need better filtered search, quantization, or scale past a single Postgres instance. Use Pinecone if you want zero operational responsibility and can accept pricing and vendor lock-in. Use Chroma only for prototyping." Youngju: "Laptop or single-node RAG prototype: Chroma, LanceDB, DuckDB VSS, sqlite-vec. Monolithic SaaS under 100M vectors, Postgres already there: pgvector + pgvectorscale. K8s self-host 100M to 1B vectors: Qdrant, Weaviate, Milvus. Minimal ops managed: Pinecone Serverless, Qdrant Cloud. Above 1B vectors GPU indexing: Milvus GPU, Vespa, FAISS." SystemsInternals: "Pinecone: managed, serverless. Weaviate: semantic + keyword hybrid. Qdrant: high performance, self-hosted. Milvus: highest scale, cloud-native. pgvector: simpler stacks, existing Postgres."

How does metadata filtering work in vector databases?

Vector databases combine ANN vector search with metadata filtering using pre-filtering, post-filtering, or in-graph filtering. StochasticSandbox: "Qdrant uses a payload index combined with a filterable HNSW. During graph traversal, it checks filter conditions on each visited node and skips non-matching nodes. When the filter is very selective (<1% match), it falls back to a pre-filtered set with brute-force search. Weaviate uses a roaring bitmap intersection approach with inverted indexes on metadata properties." SystemsInternals: "Pinecone keeps payload index. Most production deployments combine HNSW with PQ and a re-ranking pass." Filtering approaches: (1) Pre-filtering: filter metadata first, then search only matching vectors. Best for very selective filters. (2) Post-filtering: search all vectors, then filter results. Simple but may return fewer than k results. (3) In-graph filtering: check filter conditions during graph traversal. Best balance of recall and speed. Qdrant's adaptive approach handles all cases well.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →