GraphRAG for Enterprise AI: Beyond Vector Search

TL;DR — GraphRAG combines knowledge graphs with retrieval-augmented generation to answer questions that vector search cannot. It extracts entities and relationships from text, builds a knowledge graph, clusters entities into hierarchical communities, and generates summaries at each level. At query time, it traverses the graph for multi-hop reasoning. GraphRAG achieves 3.4x the accuracy of vector RAG on multi-hop queries and 70-80% win rate on global synthesis questions. However, it costs ~10x more to index and performs worse on simple factual lookups. The practical enterprise pattern is hybrid: vector search for factual queries, graph traversal for relationship queries. Microsoft's open-source GraphRAG implementation provides the reference architecture. For domains with rich relational structure (financial services, healthcare, supply chain), GraphRAG is the difference between AI that finds documents and AI that understands connections.

Vector RAG finds documents that sound similar to your question. GraphRAG finds documents that are connected to your question. This distinction matters for enterprise AI because business questions are rarely about finding a single passage — they're about understanding relationships.

"What's the risk profile of this customer?" requires connecting the customer record to their subsidiaries, their executives' backgrounds, relevant regulatory actions, and recent market events. Vector search retrieves documents that mention "risk" and "customer." GraphRAG traverses the relationships between the customer entity and all connected entities, returning a synthesized answer that no single document contains.

Microsoft's GraphRAG research showed a 70-80% win rate over naive RAG on comprehensiveness and diversity for global questions. A head-to-head comparison on financial compliance documents confirmed: GraphRAG scores 31.3% higher on multi-hop queries, while vector RAG scores 9.1% higher on single-hop queries.

What Problems Does Vector RAG Fail On?

Vector RAG fails on three categories of questions:

Multi-hop queries require connecting information across multiple documents. "Who is the CTO of the company that acquired our former supplier?" requires traversing: supplier → acquisition → acquiring company → CTO. Vector search retrieves documents mentioning "CTO," "acquisition," and "supplier" separately but cannot connect them.

Global queries require reasoning about an entire corpus. "What are the main themes across all our customer complaints?" requires synthesizing thousands of documents. Vector search retrieves the top-k most similar chunks, missing the broader picture.

Relationship queries require understanding how entities connect. "Which regulations apply to products containing component X?" requires traversing: product → contains → component → regulated by → regulation. Vector search has no concept of relationships.

How Does GraphRAG Work?

GraphRAG has two phases: indexing and querying.

Indexing Phase

Phase 1: Text Segmentation
Documents are split into TextUnits — chunks of text that serve as the unit of analysis. Each TextUnit retains a reference to its source document for citation.

Phase 2: Entity and Relationship Extraction
An LLM processes each TextUnit to extract entities (people, organizations, places, events) and relationships (connections between entities with descriptive context). Each entity gets a title, type, and description. Each relationship gets a source, target, and description.

# Simplified entity extraction
extracted = llm.extract("""
    Acme Corp acquired BetaSoft on March 15, 2025. 
    John Smith is the CEO of Acme Corp.
    BetaSoft's flagship product is DataPipeline Pro.
""")

# Result:
# Entities:
#   - Acme Corp (Organization)
#   - BetaSoft (Organization)
#   - John Smith (Person)
#   - DataPipeline Pro (Product)
#
# Relationships:
#   - Acme Corp -> acquired -> BetaSoft (March 15, 2025)
#   - John Smith -> CEO of -> Acme Corp
#   - BetaSoft -> produces -> DataPipeline Pro

Phase 3: Graph Construction
Entity descriptions from multiple TextUnits are consolidated. If "Acme Corp" appears in 50 documents, the descriptions are merged into a single comprehensive description. The graph is built with entities as nodes and relationships as edges.

Phase 4: Community Detection
The hierarchical Leiden algorithm clusters the entity graph into communities at multiple levels. Top-level communities represent broad themes. Lower-level communities represent specific topics within those themes. This creates a navigable hierarchy from high-level overview to detailed local clusters.

Phase 5: Community Summarization
An LLM generates a summary for each community, creating a hierarchical understanding of the dataset. Top-level community summaries provide a global overview. Lower-level summaries provide detailed local context. These pre-computed summaries reduce query-time LLM costs.

Query Phase

GraphRAG supports four query modes:

class GraphRAGQueryRouter:
    """Routes queries to the appropriate GraphRAG search mode."""

    def route(self, query: str, query_type: str) -> str:
        if query_type == "global":
            return self.global_search(query)
        elif query_type == "local":
            return self.local_search(query)
        elif query_type == "drift":
            return self.drift_search(query)
        else:
            return self.basic_search(query)

    def global_search(self, query: str) -> str:
        """Answer holistic questions using community summaries."""
        # Dynamic community selection: rate relevance of each community
        # report, traverse to children if relevant, skip if not
        relevant_communities = self._select_relevant_communities(query)

        # Map: generate partial answers from each relevant community
        partial_answers = [
            self._map_community(query, community)
            for community in relevant_communities
        ]

        # Reduce: synthesize final answer from partial answers
        return self._reduce_answers(query, partial_answers)

    def local_search(self, query: str) -> str:
        """Answer entity-specific questions by fanning out."""
        # Find the most relevant entities
        entities = self._find_entities(query)

        # Fan out to connected entities, relationships, and text units
        context = self._fan_out(entities)

        # Generate answer with graph context
        return self._synthesize(query, context)

Dynamic Community Selection

Microsoft's improved global search uses dynamic community selection: starting from the root of the knowledge graph, an LLM rates how relevant each community report is to the query. Irrelevant communities and their sub-communities are pruned. Relevant communities are traversed downward. Only relevant reports reach the map-reduce operation, reducing token cost and improving response quality.

GraphRAG vs Vector RAG: When to Use Each

The practical enterprise pattern is hybrid RAG: vector search handles the majority of factual queries, while graph traversal handles the subset requiring relationship reasoning. A query router classifies each query and sends it to the appropriate pipeline.

Enterprise Use Cases

Financial Services: Risk Assessment

Understanding a company's risk profile requires traversing: subsidiaries, executive backgrounds, regulatory actions, peer comparisons, and market events. These relationships span thousands of documents. GraphRAG represents them as a navigable graph.

Healthcare: Clinical Pathways

"What treatments are contraindicated for patients with condition X who are taking medication Y?" requires traversing: patient condition → medication → drug interactions → contraindications. Vector search retrieves documents mentioning the condition and medication separately. GraphRAG traverses the relationship graph.

Supply Chain: Compliance

"Which regulations apply to products containing component X?" requires traversing: product → contains → component → regulated by → regulation → jurisdiction. GraphRAG handles this multi-hop traversal naturally.

Legal: Case Research

"Which precedents are relevant to this case given the jurisdiction and subject matter?" requires connecting: case → jurisdiction → subject matter → relevant precedents → rulings. GraphRAG traverses the citation graph.

How to Implement GraphRAG

Using Microsoft's Open-Source GraphRAG

Microsoft's GraphRAG is the reference implementation. It provides:

graphrag_config:
  # Entity extraction
  entity_extraction:
    model: "gpt-4o"  # or local LLM
    entity_types: ["organization", "person", "location", "event"]
    max_entities_per_chunk: 10

  # Community detection
  community_detection:
    algorithm: "hierarchical_leiden"
    max_cluster_size: 10
    levels: 3  # number of hierarchy levels

  # Community summarization
  summarization:
    model: "gpt-4o"
    max_tokens: 1500
    summary_length: "detailed"

  # Query settings
  query:
    global_search:
      max_communities: 50
      dynamic_selection: true
    local_search:
      max_entities: 5
      fan_out_depth: 2

Self-Hosted GraphRAG Architecture

For enterprise deployment, GraphRAG components map to self-hosted infrastructure:

A company brain architecture naturally incorporates GraphRAG: the knowledge graph layer handles relationship queries while the vector store handles factual lookups. The hybrid retrieval pattern extends to include graph traversal as a third retrieval channel.

GraphRAG Implementation Checklist

• [ ] Analyze query distribution: what percentage requires multi-hop reasoning?
• [ ] If <20% multi-hop: use hybrid retrieval (vector + BM25)
• [ ] If >30% multi-hop: invest in GraphRAG infrastructure
• [ ] Select entity extraction model (local LLM for self-hosted, API for cloud)
• [ ] Define entity types relevant to your domain (organization, person, regulation, product)
• [ ] Configure relationship extraction prompts for domain-specific relationships
• [ ] Deploy knowledge graph database (Neo4j, FalkorDB, Apache AGE)
• [ ] Implement hierarchical Leiden community detection
• [ ] Generate community summaries at each hierarchy level
• [ ] Implement query router (global, local, DRIFT, basic)
• [ ] Configure dynamic community selection for global search
• [ ] Set up graph re-indexing pipeline for incremental updates
• [ ] Implement hybrid mode: vector search for simple queries, graph for complex
• [ ] Benchmark GraphRAG vs vector RAG on your actual query distribution
• [ ] Measure answer quality with RAG Triad (context relevance, groundedness, answer relevance)
• [ ] Monitor indexing costs (LLM calls for entity extraction and summarization)
• [ ] Integrate with company brain architecture
• [ ] Enforce document-level ACLs in graph traversal
• [ ] Test with real enterprise queries (compliance, risk, supply chain)
• [ ] Evaluate RAG vs knowledge graphs tradeoffs for your use case

FAQ

What is GraphRAG?

GraphRAG is a retrieval-augmented generation approach that uses knowledge graphs instead of (or alongside) vector similarity search. It extracts entities and relationships from text, builds a knowledge graph, clusters entities into hierarchical communities, and generates community summaries. At query time, it traverses the graph to find connected information across documents. Microsoft's GraphRAG implementation showed 70-80% win rate over naive RAG on comprehensiveness and diversity metrics for global questions.

When should you use GraphRAG instead of vector RAG?

Use GraphRAG when your queries require multi-hop reasoning (connecting information across multiple documents), relationship awareness (understanding how entities connect), or global synthesis (answering questions about an entire corpus). Use vector RAG for simple factual lookups where the answer is in a single passage. Approximately 35% of enterprise business queries require multi-hop reasoning — in financial services, healthcare, and manufacturing, that number is closer to 55%.

How does GraphRAG indexing work?

GraphRAG indexing has five phases: (1) segment text into TextUnits, (2) use an LLM to extract entities and relationships from each TextUnit, (3) consolidate entity descriptions and build the graph, (4) apply hierarchical Leiden clustering to detect communities, (5) generate LLM summaries for each community. The output is a knowledge graph with hierarchical community summaries that enable both local and global queries.

What are the query modes in GraphRAG?

GraphRAG supports four query modes: Global Search (reasoning about holistic questions using community summaries), Local Search (reasoning about specific entities by fanning out to neighbors), DRIFT Search (combining entity fan-out with community context), and Basic Search (standard vector search for simple queries). The query router selects the appropriate mode based on query type.

How much does GraphRAG cost compared to vector RAG?

GraphRAG indexing costs approximately 10x more than vector RAG due to LLM-based entity extraction and community summarization. However, query-time costs can be lower for global questions because community summaries are pre-computed (2-3% of token cost vs. full corpus summarization). The economic decision depends on your query distribution: if more than 30% of queries require multi-hop reasoning, the indexing investment pays for itself in answer quality.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →