KAG vs RAG: Benchmarking Knowledge Graphs Against Vector Retrieval
KAG vs RAG: Benchmarking Knowledge Graphs Against Vector Retrieval
Two pipelines. One question set. Quantified results.
RAG is now the default pattern for grounding LLM responses in external knowledge. KAG challenges that assumption by replacing chunk retrieval with graph traversal. This project builds both from scratch, runs them against the same queries under identical conditions, and measures the difference.
Why This Comparison Matters
The standard RAG pipeline is well understood: embed your documents into a vector store, retrieve the top-k chunks most similar to a query, and pass them as context to the LLM. For straightforward factual lookups where the answer exists inside a single passage, this works reliably.
The failure mode appears with multi-hop reasoning—questions that require chaining facts across multiple documents. Consider the query:
“The AI that beat the world Go champion was created by which lab, and what large language model did that lab later build?”
Answering this correctly requires connecting AlphaGo → DeepMind → Gemini across document boundaries. RAG retrieves independent chunks with no mechanism to follow relationship chains. It finds fragments but cannot join them coherently.
Knowledge-Augmented Generation (KAG) addresses this by constructing a knowledge graph during ingestion. Named entities are extracted, their relationships identified, and everything stored as a graph in Neo4j. At query time, KAG links entities in the question to graph nodes, traverses their relationships, and merges that structured context with standard vector-retrieved chunks before prompting the LLM.
Architecture Overview
RAG Pipeline
Documents
↓
Text Chunking (sliding window)
↓
Embedding (NVIDIA NIM)
↓
Vector Store (in-memory FAISS)
↓
Cosine Similarity Retrieval (top-k)
↓
LLM Prompt Construction
↓
ResponseThe RAG implementation follows a clean, minimal design. Chunks are generated with a configurable window size and overlap. NVIDIA NIM embeddings produce the vector representations. At query time, the query is embedded, cosine similarity is computed against all chunk vectors, and the top results are assembled into a context block for the LLM.
KAG Pipeline
Documents
↓
Named Entity Recognition (SpaCy)
↓
Relationship Extraction (LLM-assisted)
↓
Graph Ingestion (Neo4j)
↓
Entity Linking (query → graph nodes)
↓
Graph Traversal (multi-hop)
↓
Context Merge (graph + vector chunks)
↓
LLM Prompt Construction
↓
ResponseKAG’s ingestion step is heavier than RAG’s. SpaCy extracts named entities from each document. An LLM pass identifies relationships between co-occurring entities. The resulting entity-relationship triples are stored in Neo4j as a property graph. At query time, entities in the question are linked to graph nodes using exact and fuzzy matching, and a Cypher traversal collects their immediate neighbours. This structured context is merged with standard vector-retrieved chunks to give the LLM both relational and passage-level evidence.
Benchmark Results
The evaluation ran six multi-hop test queries against both pipelines using identical documents and model configurations. Each response was scored by an LLM judge on four dimensions: factual accuracy, hallucination rate, context relevance, and latency.
| Metric | RAG | KAG | Winner |
|---|---|---|---|
| Factual Accuracy | 67.4% | 84.2% | KAG |
| Hallucination Rate | 34.7% | 15.8% | KAG |
| Context Relevance | 51.2% | 51.2% | Tied |
| Avg Latency | 2993ms | 1691ms | KAG |
KAG leads on three of four metrics, with the largest margin on factual accuracy (+16.8 pp) and hallucination rate (-18.9 pp). The latency advantage for KAG is counterintuitive at first—graph traversal is typically assumed to add overhead—but in practice the structured context reduces the LLM’s need to reason across ambiguous fragments, shortening the generation.
Context relevance is tied because both pipelines ultimately retrieve passages from the same document set. The difference is in how those passages are combined with structured evidence, not in passage selection quality.
LLM-as-Judge Scoring
Manual evaluation at scale is expensive. This project automates evaluation using an LLM judge that receives the question, both pipeline responses, and a reference answer, then scores each response numerically.
The judge prompt is structured to elicit independent evaluation rather than comparison. Each dimension is scored separately with explicit criteria:
- Factual accuracy: Claims that are directly verifiable against the reference
- Hallucination rate: Claims that contradict the reference or introduce unsupported information
- Context relevance: How well the retrieved context maps to the question’s information need
- Latency: Wall-clock time from query submission to response completion
Scores are normalised to a 0–100 scale and aggregated across all queries to produce the summary table. The raw per-query breakdowns are available in the web dashboard.
Web Dashboard
The project ships with a self-contained web dashboard served at http://localhost:8000. It has four sections:
- Project overview — architecture diagrams and a summary of the methodology
- Evaluation methodology — explanation of the judge prompt, scoring criteria, and benchmark design
- Benchmark results — per-query breakdowns with the summary table and a visual comparison
- Interactive playground — a query interface where you can submit your own questions, see both pipelines respond in real time, and get live LLM-as-Judge scores on the spot
The playground is the most useful part for development. Seeing RAG and KAG respond to the same question side-by-side, with scores attached, makes the trade-offs immediately visible rather than abstract.
Implementation Notes
Entity Extraction
SpaCy’s en_core_web_sm model handles initial NER. The model is deliberately lightweight—the goal is entity identification, not high-recall extraction. A secondary LLM pass refines entity boundaries and extracts relationship types that SpaCy’s rule-based system misses.
Graph Schema
The Neo4j graph uses a minimal schema: Entity nodes with name and type properties, connected by RELATED_TO edges with a relation property storing the extracted relationship label. This keeps the schema generic across document types while preserving enough structure for meaningful traversal.
Retrieval Merging
The context merge step concatenates graph-derived context (formatted as structured triples) with vector-retrieved passages. A simple priority scheme places graph context first when entity overlap with the query is high, falling back to passage-heavy context for queries where entity linking fails.
Query Entity Linking
Entity linking is the most fragile component. The current implementation uses exact string matching with a lowercase normalisation fallback. Queries that rephrase entity names differently from how they appear in the documents will fail to link, silently degrading to RAG-only behaviour. Fuzzy matching and embedding-based entity resolution are the obvious next steps.
Where Each Pipeline Wins
RAG is better when:
- The answer exists in a single coherent passage
- Documents are long and entity-rich, making graph construction expensive
- Latency budget is tight and ingestion time matters
- The question is simple and factual without relational depth
KAG is better when:
- Answering requires connecting facts across document boundaries
- Questions involve named entities and their relationships
- Hallucination rate must be minimised (structured evidence anchors the LLM)
- The document set is stable enough to justify graph construction overhead
Multi-hop reasoning is the decisive criterion. For single-hop factual lookup, both pipelines perform similarly. For questions that require chaining—person to organisation to product, event to cause to consequence—KAG’s structured traversal provides evidence that RAG’s independent chunk retrieval cannot replicate.
Design Trade-Offs
| Decision | Benefit | Trade-Off |
|---|---|---|
| SpaCy for NER | Fast, no API cost, runs offline | Lower recall than LLM-based extraction |
| In-memory vector store | Zero infrastructure overhead | Does not scale beyond memory limits |
| Minimal graph schema | Portable across document types | Loses domain-specific relationship richness |
| LLM-as-Judge for evaluation | Scalable, consistent scoring | Judge quality bounded by model capability |
| Context merge (graph + vector) | Combines strengths of both approaches | Adds prompt length and complexity |
Stack
- Language: Python 3.10+
- Embeddings + LLM: NVIDIA NIM API
- Entity Extraction: SpaCy (
en_core_web_sm) - Graph Database: Neo4j (Desktop or Community Server)
- Vector Retrieval: Custom FAISS-style cosine similarity
- Web Dashboard: FastAPI + vanilla HTML/CSS/JS
Repository
Full implementation, setup instructions, and benchmark data:
https://github.com/JashT14/Kag_vs_Rag
Final Note
RAG is not broken. It is optimised for a specific retrieval pattern that handles the majority of real-world use cases well. KAG is not a replacement—it is an extension that addresses the subset of queries where relational structure matters more than passage similarity.
Building both from scratch and measuring them against the same dataset is the clearest way to understand the trade-off. The numbers in this benchmark are not arguments for abandoning RAG. They are a map of when it is worth the additional complexity of graph construction.