Embedding Dimensions: 384 vs 768 vs 1024 vs 1536 vs 3072 Explained

TL;DR — Embedding dimensions: 384 for prototyping, 768-1024 as production default, 1536 for OpenAI, 3072 for max quality. PragmaticByHarsh: "Start with 768-1024 dimensions. Modern models support Matryoshka truncation — cut dimensions in half with minimal quality loss. Quality doesn't scale linearly. Voyage-4-large at 1024-dim scores 72.3% MTEB vs OpenAI 3-large at 3072-dim 64.6%." CodeSOTA: "Default to 768 — best quality/cost ratio. Prototype at 384, deploy at 768+. Past 1024, gains are marginal. At 100M docs: 384d = 150GB vs 3072d = 1.2TB — 8x cost multiplier." AITLDR: "Under 1M vectors, 1536 is safe. At tens of millions, 768-1024 is practical. MTEB: 384d ~56, 768d ~63, 1024d ~64, 1536d ~64.6, 3072d ~65. Gains compress quickly." TypeGraph: "halfvec beats vector in pgvector — cuts storage in half while preserving quality. 1024d halfvec: nDCG 0.658, recall 0.92. Prefer halfvec unless benchmark proves float32 is worth it." Decompressed: "Start with text-embedding-3-small. MRL: cut 1536 to 512 dims, lose 2-4 MRR points, save two-thirds storage." Learn more with embeddings, embedding models, cosine similarity, and pgvector tutorial.

CodeSOTA explains the fundamentals: "Embedding dimensions like 384, 768, and 1024 look arbitrary — but they are the product of transformer architecture constraints, GPU hardware alignment, and information-theoretic bounds. Doubling the embedding dimension doubles memory, roughly doubles search latency for brute-force retrieval — but the quality gains diminish sharply."

PragmaticByHarsh provides the practical guidance: "Start with 768-1024 dimensions. This range balances quality, cost, and speed for most production systems. Modern embedding models support dimension reduction via Matryoshka learning. You can often cut dimensions in half with minimal quality loss, test this before defaulting to maximum dimensions."

Embedding Dimensions Architecture

Dimension Comparison Table

Storage Cost at Scale

MRL Truncation Quality Retention

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class DimensionPreset(Enum):
    PROTOTYPE = 384
    PRODUCTION = 768
    RAG_OPTIMIZED = 1024
    OPENAI_SMALL = 1536
    MAX_QUALITY = 3072

@dataclass
class EmbeddingDimensionSelector:
    """Select and optimize embedding dimensions for production RAG."""

    def recommend(self, vector_count: int,
                  precision_required: str = "standard",
                  storage_budget_gb: float = None,
                  latency_budget_ms: float = None,
                  use_matryoshka: bool = True,
                  use_halfvec: bool = True) -> dict:
        """Recommend embedding dimensions based on constraints."""

        if vector_count < 100_000 and precision_required == "low":
            return self._result(384, "Prototyping: 384 dims, "
                              "fastest, lowest storage")

        if storage_budget_gb and vector_count > 0:
            max_dims_float32 = int(
                (storage_budget_gb * 1e9) /
                (vector_count * 4))
            max_dims_halfvec = int(
                (storage_budget_gb * 1e9) /
                (vector_count * 2))

            if use_halfvec and max_dims_halfvec >= 1024:
                return self._result(1024, 
                    "1024 dims with halfvec: fits budget, "
                    "best quality/cost",
                    use_halfvec=True)
            elif max_dims_float32 >= 768:
                return self._result(768,
                    "768 dims: fits storage budget, "
                    "production default")
            elif max_dims_float32 >= 384:
                return self._result(384,
                    "384 dims: fits storage budget, "
                    "prototyping quality")

        if latency_budget_ms and latency_budget_ms < 30:
            return self._result(384,
                "384 dims: meets latency budget "
                "(<10ms query)")

        if precision_required == "high":
            if use_matryoshka:
                return self._result(1024,
                    "1024 dims with MRL: high precision, "
                    "can truncate if needed",
                    use_matryoshka=True)
            return self._result(1536,
                "1536 dims: high precision, "
                "OpenAI 3-small default")

        if precision_required == "maximum":
            return self._result(3072,
                "3072 dims: maximum quality, "
                "test if improvement justifies cost")

        return self._result(768,
            "768 dims: production default, "
            "best quality/cost ratio")

    def get_storage_estimate(self, dimensions: int,
                             vector_count: int,
                             use_halfvec: bool = False) -> dict:
        """Estimate storage requirements."""
        bytes_per_float = 2 if use_halfvec else 4
        raw_bytes = dimensions * bytes_per_float * vector_count
        hnsw_overhead = raw_bytes * 2.5  # HNSW adds ~2.5x

        return {
            "dimensions": dimensions,
            "vector_count": vector_count,
            "storage_type": "halfvec" if use_halfvec else "vector",
            "bytes_per_vector": dimensions * bytes_per_float,
            "raw_storage_gb": raw_bytes / 1e9,
            "with_hnsw_gb": hnsw_overhead / 1e9,
            "monthly_cost": (
                hnsw_overhead / 1e9 * 0.15
            ),  # $0.10-0.20/GB/month
        }

    def get_mrl_truncation_guide(self,
                                 full_dims: int = 1536) -> dict:
        """Matryoshka Representation Learning truncation guide."""
        truncations = [
            (768, 100.0, 0, "Full dimension (if 768+)"),
            (512, 99.2, -0.5, "Minimal loss"),
            (384, 98.1, -1.2, "Great tradeoff"),
            (256, 96.5, -2.3, "Good for prototyping"),
            (128, 92.8, -4.8, "Noticeable degradation"),
            (64, 85.4, -9.7, "Topic-level only"),
        ]

        return {
            "full_dimensions": full_dims,
            "supported_models": [
                "OpenAI text-embedding-3-small (1536)",
                "OpenAI text-embedding-3-large (3072)",
                "nomic-embed-text-v1.5 (768)",
                "Cohere embed-v4 (1024)",
                "Voyage-4 (1024)",
                "gemini-embedding-001 (3072)",
            ],
            "truncation_options": [
                {
                    "dimensions": t[0],
                    "quality_retained": t[1],
                    "mteb_delta": t[2],
                    "note": t[3],
                    "storage_savings": (
                        1 - t[0] / full_dims
                    ),
                }
                for t in truncations
                if t[0] <= full_dims
            ],
            "notes": (
                "MRL allows truncation without retraining. "
                "Request fewer dimensions from same model. "
                "text-embedding-3-large at 256 dims "
                "outperforms ada-002 at 1536 dims. "
                "6x reduction with better quality."
            ),
        }

    def get_halfvec_comparison(self,
                               dimensions: int = 1024,
                               vector_count: int = 1_000_000) -> dict:
        """Compare vector vs halfvec storage and quality."""
        float32_bytes = dimensions * 4 * vector_count
        halfvec_bytes = dimensions * 2 * vector_count

        return {
            "dimensions": dimensions,
            "vector_count": vector_count,
            "vector_float32": {
                "bytes_per_vector": dimensions * 4,
                "total_storage_gb": float32_bytes / 1e9,
                "precision": "32-bit float",
            },
            "halfvec": {
                "bytes_per_vector": dimensions * 2,
                "total_storage_gb": halfvec_bytes / 1e9,
                "precision": "16-bit float",
                "storage_reduction": "50%",
            },
            "quality_comparison": {
                "vector_ndcg": 0.6550,
                "halfvec_ndcg": 0.6580,
                "vector_recall": 0.9100,
                "halfvec_recall": 0.9200,
                "verdict": "halfvec matches or slightly "
                           "exceeds vector quality",
            },
            "recommendation": (
                "Prefer halfvec over vector unless "
                "benchmark proves float32 is worth "
                "the extra storage. halfvec cuts "
                "storage in half while preserving "
                "retrieval quality."
            ),
        }

    def _result(self, dims: int, reason: str,
                use_halfvec: bool = False,
                use_matryoshka: bool = False) -> dict:
        return {
            "recommended_dimensions": dims,
            "reason": reason,
            "use_halfvec": use_halfvec,
            "use_matryoshka": use_matryoshka,
            "memory_per_vector_kb": round(dims * 4 / 1024, 1),
            "latency_estimate_ms": self._estimate_latency(dims),
        }

    def _estimate_latency(self, dims: int) -> str:
        if dims <= 256:
            return "<5ms"
        elif dims <= 384:
            return "<10ms"
        elif dims <= 512:
            return "10-20ms"
        elif dims <= 768:
            return "10-30ms"
        elif dims <= 1024:
            return "30-50ms"
        elif dims <= 1536:
            return "50-100ms"
        else:
            return "100-500ms"

Embedding Dimensions Checklist

• [ ] Embedding dimensions are the number of values used to represent a chunk, document, or query
• [ ] 384 dimensions: 1.5 KB/vector, MTEB ~56, fastest, for prototyping and edge
• [ ] 768 dimensions: 3 KB/vector, MTEB ~63, fast, production default, best quality/cost
• [ ] 1024 dimensions: 4 KB/vector, MTEB ~64-72, moderate, RAG-optimized models (BGE-M3, Voyage-4)
• [ ] 1536 dimensions: 6 KB/vector, MTEB ~62-64.6, slower, OpenAI 3-small default
• [ ] 3072 dimensions: 12 KB/vector, MTEB ~65, slowest, max quality, dense technical text
• [ ] Start with 768-1024 dimensions for most production systems
• [ ] Prototype at 384, deploy at 768+ — dimension is a tuning knob, not a fixed choice
• [ ] Past 1024, gains are marginal — spend budget on better data instead
• [ ] Quality doesn't scale linearly with dimensions — training matters more than size
• [ ] Voyage-4-large at 1024-dim scores 72.3% MTEB vs OpenAI 3-large at 3072-dim 64.6%
• [ ] A well-trained 1024-dim model can beat a poorly-trained 3072-dim model
• [ ] Storage scales linearly: 10M vectors at 384d = 15GB, at 3072d = 120GB (8x cost)
• [ ] At 100M docs: 384d = 150GB vs 3072d = 1.2TB — 8x infrastructure cost multiplier
• [ ] Vector database storage costs $0.10-0.20/GB/month
• [ ] HNSW index adds ~2.5x on top of raw vector storage
• [ ] Latency by dimensions: 256 (<5ms), 384 (<10ms), 512 (10-20ms), 768 (10-30ms), 1024 (30-50ms), 1536 (50-100ms), 3072 (100-500ms)
• [ ] Matryoshka Representation Learning (MRL) allows truncation without retraining
• [ ] MRL supported by: OpenAI 3-series, Cohere v4, Voyage v4, gemini-embedding-001, nomic-embed-text
• [ ] MRL truncation: 768 (100% quality), 512 (99.2%), 384 (98.1%), 256 (96.5%), 128 (92.8%)
• [ ] text-embedding-3-large at 256 dims outperforms ada-002 at 1536 dims — 6x reduction with better quality
• [ ] Cut text-embedding-3-small from 1536 to 512: lose 2-4 MRR points, save two-thirds storage
• [ ] Use MRL truncation when storage or ANN speed is a constraint
• [ ] Measure accuracy penalty before assuming truncation is acceptable
• [ ] pgvector halfvec cuts storage roughly in half while preserving retrieval quality
• [ ] 1024d vector: ~4KB, 1024d halfvec: ~2KB — 50% storage reduction
• [ ] halfvec reduction also matters for HNSW index size (less data to keep hot)
• [ ] Test results: 1024d halfvec nDCG 0.658, recall 0.92 — slightly exceeds float32 vector
• [ ] Prefer pgvector halfvec over vector unless benchmark proves float32 is worth extra storage
• [ ] Use halfvec when: storage cost matters, HNSW memory constrained, >1M vectors
• [ ] Quality/KB efficiency: 384d (37.3), 768d (21.0), 1024d (16.0), 1536d (10.8), 3072d (5.4)
• [ ] 768 is popular because you get most quality at a fraction of cost
• [ ] Under 1M vectors: 1536 dimensions is a safe default
• [ ] At tens of millions of vectors: 768-1024 dimensions more practical
• [ ] For very high precision (multilingual, dense technical): 3072 may justify cost
• [ ] Always benchmark recall on a sample of your own data
• [ ] Start with text-embedding-3-small — fast, cheap, reliable for general retrieval
• [ ] Test text-embedding-3-large if precision matters and cost is secondary
• [ ] If improvement is 3-4 MRR points, 6.5x cost increase might be worth it
• [ ] If improvement is 1 MRR point, it isn't worth the cost
• [ ] Dimensions are determined by transformer architecture (heads × head_dim)
• [ ] 384 = 6 heads × 64, 768 = 12 heads × 64, 1024 = 16 heads × 64, 1536 = 24 heads × 64
• [ ] Read embeddings for fundamentals
• [ ] Read embedding models for model selection
• [ ] Read cosine similarity for distance metrics
• [ ] Read pgvector tutorial for setup with halfvec
• [ ] Read what is RAG for RAG architecture
• [ ] Read vector databases for storage
• [ ] Test: 768 dims achieves ~63 MTEB with 3KB/vector
• [ ] Test: MRL truncation from 1536 to 384 retains 98.1% quality
• [ ] Test: halfvec matches float32 quality at half storage
• [ ] Test: storage estimate matches actual index size
• [ ] Test: recommendation logic selects correct dimensions per constraints
• [ ] Document dimension choice, MRL strategy, halfvec usage, and benchmark results

FAQ

What embedding dimensions should you choose for production RAG?

Start with 768-1024 dimensions for most production systems — this range balances quality, cost, and speed. PragmaticByHarsh: "Start with 768-1024 dimensions. This range balances quality, cost, and speed for most production systems. Modern embedding models support dimension reduction via Matryoshka learning. You can often cut dimensions in half with minimal quality loss." Decompressed: "Start with text-embedding-3-small. It is fast, cheap, and reliable on general English-language retrieval. For most teams, most of the time, it is the right answer." CodeSOTA: "Default to 768 — best quality/cost ratio for most applications. Use Matryoshka models — train once, truncate to any dimension at inference. Prototype at 384, deploy at 768+. Past 1024, gains are marginal — spend your budget on better data instead." AITLDR: "Under 1 million vectors, 1536 dimensions is a safe default. At tens of millions, 768-1024 is more practical. For very high precision, 3072 may justify the cost. Always benchmark recall on your own data."

How do embedding dimensions affect storage cost and latency?

Storage and latency scale linearly with dimensions. Going from 768 to 3072 dimensions quadruples storage and significantly increases search latency. AITLDR: "Every vector in your database scales with dimensions. Going from 768 to 3072 quadruples raw storage. At 10 million vectors: 230 MB at 384 dims vs 3.7 GB at 3072 dims in float32. Cloud vector database pricing follows storage." CodeSOTA: "At 100M documents, difference between 384d and 3072d is 150 GB vs 1.2 TB — an 8x cost multiplier. Doubling dimensions roughly doubles search latency for brute-force retrieval, but quality gains diminish sharply." PragmaticByHarsh: "Vector database storage typically costs $0.10-0.20/GB/month. At 10M documents with 1536-dim: $6-12/month. Cut to 768-dim: $3-6/month." Latency by dimensions: 256 (<5ms), 384 (<10ms), 512 (10-20ms), 768 (10-30ms), 1024 (30-50ms), 1536 (50-100ms), 3072 (100-500ms).

What is Matryoshka Representation Learning (MRL) and how does it help?

MRL allows you to truncate embedding dimensions without retraining — you can request fewer dimensions from the same model with minimal quality loss. PragmaticByHarsh: "Modern embedding models (OpenAI 3-series, Cohere v4, Voyage v4, gemini-embedding-001) support Matryoshka Representation Learning. You can reduce dimensions without retraining. text-embedding-3-large at 256 dimensions outperforms ada-002 at 1536 dimensions on MTEB. That is a 6x reduction in size with better quality." Decompressed: "Models trained with MRL produce vectors you can truncate to a smaller size without proportional quality loss. Both text-embedding-3-small and text-embedding-3-large support MRL: cut text-embedding-3-small from 1536 to 512 dimensions and lose roughly 2-4 MRR points while saving two-thirds of storage." CodeSOTA: "Use Matryoshka models — train once, truncate to any dimension at inference. Prototype at 384, deploy at 768+ — dimension is a tuning knob, not a fixed choice." MRL truncation quality: 768 (100%), 512 (99.2%), 384 (98.1%), 256 (96.5%), 128 (92.8%).

Should you use pgvector halfvec instead of vector for embeddings?

Yes, halfvec cuts storage roughly in half while preserving retrieval quality, making it the preferred choice for most pgvector deployments. TypeGraph: "halfvec beats vector when using pgvector on Postgres because it cuts raw vector storage roughly in half while preserving retrieval quality. A 1024-dimensional vector is roughly 4KB. A 1024-dimensional halfvec is roughly 2KB. That reduction also matters for HNSW index size. Prefer pgvector halfvec over vector unless your benchmark proves float32 is worth the extra storage." Test results: 1024 dims vector nDCG@10: 0.6550, recall@10: 0.9100. 1024 dims halfvec nDCG@10: 0.6580, recall@10: 0.9200 — halfvec actually scored slightly higher. Use halfvec when: (1) Storage cost matters. (2) HNSW index memory is a constraint. (3) You have >1M vectors. (4) Your benchmark shows no significant quality loss.

Do higher embedding dimensions always mean better quality?

No — quality doesn't scale linearly with dimensions. A well-trained 1024-dim model can beat a poorly-trained 3072-dim model. Training matters more than size. PragmaticByHarsh: "Quality does not scale linearly with dimensions. A well-trained 1024-dim model can beat a poorly-trained 3072-dim model. Training matters more than size. Voyage-4-large at 1024-dim scores 72.3% MTEB. OpenAI text-embedding-3-large at 3072-dim scores 64.6% MTEB. The 1024-dim model wins because it is trained specifically for retrieval." AITLDR: "Larger default dimension does not automatically mean better MTEB score. Voyage AI voyage-3 achieves competitive quality at 1024 dimensions, costing 3-4x less storage than OpenAI 3072-dimensional model. The quality of training data and training objective matter at least as much as raw dimension count." CodeSOTA: "Past 1024, gains are marginal — spend your budget on better data instead." MTEB scores by dimension: 384 (~56), 768 (~63), 1024 (~64), 1536 (~64.6), 3072 (~65). Quality/KB efficiency: 384d (37.3), 768d (21.0), 1024d (16.0), 1536d (10.8), 3072d (5.4).

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →