Improving retrieval-augmented generation pipelines through knowledge graph integration

Introduction. Large Language Models (LLMs) have brought sweeping changes to natural language processing, yet they still struggle with notable constraints — among them, outdated knowledge cutoffs, a tendency to hallucinate, and limited access to proprietary or domain-specific data. Retrieval-Augmented Generation (RAG) tackles these shortcomings by merging neural language generation with external knowledge retrieval. Conventional RAG systems rely on vector similarity search through dense embeddings to pull relevant context from document collections. While this works well for semantic matching, it treats each document as a standalone unit, failing to account for the intricate network of relationships that typically define real-world knowledge. Consider a query like "Who is the CEO of the company that acquired Instagram?" — answering it demands following a chain of connections: Instagram → acquired by → Facebook → CEO → Mark Zuckerberg. Vector-based retrieval in RAG systems is held back by three fundamental weaknesses. First, it suffers from structural blindness — flat embeddings are simply incapable of representing hierarchical or relational structures within data. Second, multi-hop reasoning poses a persistent challenge, as queries that demand synthesizing information across several documents are difficult to handle effectively. Third, context fragmentation remains an issue, since related pieces of information spread across multiple documents rarely get consolidated in a meaningful way. Knowledge graphs (KGs) offer a natural way to encode structured relationships between entities, presenting an alternative representation that directly tackles the shortcomings outlined above. Despite this promise, the integration of KGs into RAG systems has received relatively little attention — especially when it comes to systematically comparing different integration strategies. This paper puts forward the following contributions: a structured comparison of three RAG retrieval strategies — vector search, graph-enhanced retrieval, and GNN-based embeddings; an automated pipeline for building knowledge graphs directly from unstructured text; experimental validation on a synthetic yet representative dataset that demonstrates measurable performance Development of science and technology Scientific and technical journal №2/2026 213 Informatika va axborot-kommunikatsion tizimlar gains; and an analysis of computational trade-offs alongside practical implementation considerations. The rest of this paper is structured as follows: Section II covers related work, Section III outlines the methodology, Section IV details the experimental setup, Section V walks through the results, and Section VI wraps up with conclusions and directions for future research. Related Work. RAG was originally proposed by Lewis et al. as a way to supplement language models with retrieved text passages. The architecture pairs a retriever — typically a biencoder built on BERT — with a generator such as BART or T5. Notable developments in the field include dense passage retrieval (DPR), which trains retrievers specifically for question-answering, as well as iterative retrieval methods that progressively refine results through multiple passes. Knowledge graphs have seen widespread use across NLP tasks including entity linking, relation extraction, and question answering. Graph-based QA systems such as KGQA make use of structured queries like SPARQL to interact with KGs. That said, these methods depend on wellformed queries and tend to fall short when confronted with the ambiguity inherent in natural language. More recent research has turned toward neural methods for KG reasoning. KagNet incorporates knowledge graphs to strengthen commonsense reasoning in QA tasks, while QA-GNN takes a joint reasoning approach over both text and KGs using graph neural networks, yielding notable gains on the CommonsenseQA benchmark. Graph Neural Networks have established themselves as powerful tools for learning over graph-structured data. Graph Convolutional Networks (GCNs) aggregate features from neighboring nodes through spectral convolutions, while Graph Attention Networks (GATs) build on this by introducing attention mechanisms that selectively weight neighbor contributions. GraphSAGE, meanwhile, enables inductive learning on large-scale graphs through a sampling and aggregation strategy. Within NLP, GNNs have been applied to a range of tasks including document classification, named entity recognition, and semantic role labeling. Their integration with language models continues to be an active and growing area of research. Recent efforts have started exploring how structured knowledge can be combined with neural retrieval. REALM pre-trains language models using retrieved documents, while RAGend2end takes a joint training approach for both the retriever and generator. Despite these advances, the majority of such methods still rely on vector similarity as their core mechanism, leaving graph structures largely untapped. StructGPT allows LLMs to interact with structured data via iterative prompting, though this comes at the cost of requiring multiple LLM calls. Our approach takes a different route by preencoding graph structure through GNNs, making efficient single-pass retrieval possible. Although RAG and knowledge graphs have each been studied independently, there is little research that systematically compares different strategies for weaving graph structures into RAG retrieval. Our work addresses this gap through controlled experiments on synthetic data, allowing the impact of graph-based retrieval to be assessed in isolation from confounding variables. Methodology. We introduce and evaluate three retrieval strategies for RAG systems, each progressively building on the incorporation of graph-structured knowledge. Our baseline implements standard dense retrieval RAG: - Document encoding: Each document 𝑑𝑖 is encoded into a dense vector 𝐯𝑖∈ℝ768 using a pre-trained sentence transformer (all-MiniLM-L6-v2) - Index construction: Vectors are indexed using FAISS for efficient similarity search - Query processing: Given query 𝑞, encode to 𝐯𝑞 and retrieve top-𝑘 documents by cosine similarity: sim(𝑞, 𝑑𝑖) = 𝐯𝑞⋅𝐯𝑖 ‖𝐯𝑞‖‖𝐯𝑖‖ “Fan va texnologiyalar taraqqiyoti” Ilmiy-texnikaviy jurnal №2/2026 214 Computer science and information-communication systems - Answer generation: Retrieved contexts are concatenated and fed to a language model for generation Knowledge Graph Construction. A knowledge graph is constructed from the document collection, where entities serve as nodes and relationships serve as edges. Named Entity Recognition (NER) via spaCy is applied to extract entities from the documents, with each unique entity becoming its own node. Relations are established through: - Co-occurrence: Entities appearing in the same sentence are connected - Typed relations: For synthetic data, we define explicit relations (e.g., “CEO_of”, “located_in”, “produces”) - Document linkage: Entities sharing documents are connected with weighted edges Each node is associated with: 𝐱𝑖= [𝐞𝑖; 𝐜𝑖; 𝐬𝑖] where 𝐞𝑖 is the entity name embedding, 𝐜𝑖 is aggregated context from mentions, and 𝐬𝑖 are structural features (degree, centrality). Graph-Enhanced Retrieval. The graph-enhanced approach builds on vector retrieval by incorporating graph exploration into the process: Retrieve top-𝑘 documents using vector similarity - Extract entities from query and retrieved documents - For each retrieved entity, include 𝑛-hop neighbors: 𝑁𝑛(𝑣) = {𝑢∈𝑉: 𝑑(𝑣, 𝑢) ≤𝑛} where 𝑑(𝑣, 𝑢) is the shortest path distance - Score expanded documents by combining vector similarity and graph proximity: score(𝑞, 𝑑) = 𝛼⋅sim(𝑞, 𝑑) + (1 −𝛼) ⋅graph_score(𝑞, 𝑑) The graph score considers: graph_score(𝑞, 𝑑) = ∑ ∑ 1 𝑑(𝑒𝑞, 𝑒𝑑) + 1 𝑒𝑑∈𝐸𝑑 𝑒𝑞∈𝐸𝑞 where 𝐸𝑞 and 𝐸𝑑 are entities in query and document. GNN-based Retrieval. Our GNN approach learns node representations that encode both content and graph structure: We implement a 2-layer Graph Convolutional Network: 𝐇(𝑙+1) = 𝜎(𝐃̃−1 2𝐀̃𝐃̃−1 2𝐇(𝑙)𝐖(𝑙)) where 𝐀̃ = 𝐀+ 𝐈 is the adjacency matrix with self-loops, 𝐃̃ is the degree matrix, and 𝜎 is ReLU activation. Node embedding generation: Initialize 𝐇(0) with node features; Apply GCN layers to obtain final embeddings 𝐇(2) ∈ℝ|𝑉|×𝑑; Each node embedding captures its local neighborhood structure Retrieval process: Encode query into graph space (by linking to entities); Compute similarity between query and node embeddings; Retrieve documents associated with top-ranked nodes; Use GNN embeddings for re-ranking. Training. The GNN is trained using a link prediction objective: Development of science and technology Scientific and technical journal №2/2026 215 Informatika va axborot-kommunikatsion tizimlar (𝑖,𝑗)∈𝐸 𝜎(𝐡𝑖 𝑇𝐡𝑗) −∑log (𝑖,𝑘)∉𝐸 (1 −𝜎(𝐡𝑖 𝑇𝐡𝑘)) ℒ= −∑log where positive pairs are connected nodes and negative pairs are randomly sampled. Answer Generation. Across all three approaches, an identical generation strategy is applied in order to isolate the effect of retrieval: retrieved contexts are concatenated with the query and fed into a simple template-based or extractive QA system. For the purposes of this study, the focus remains on retrieval quality rather than generation performance. Experimental Setup. A controlled synthetic dataset is created to enable rigorous evaluation, covering technology companies and products across 100 entities and 500 facts. This consists of 200 synthetic documents, each ranging from 100 to 200 words, where entities and their relationships are described through structured facts embedded in natural language. The question set includes 50 single-hop reasoning questions, 30 two-hop reasoning questions, and 20 three-hop reasoning questions, with ground truth answers annotated manually. Knowledge graph: - 100 entity nodes (companies, people, products, locations) - 400 relationship edges (founded_by, CEO_of, produces, acquired, etc.) - Average degree: 4 connections per node Evaluation Metrics: We evaluate using the following metrics: - Retrieval Accuracy: Percentage of queries where at least one relevant document is in top-k results - Mean Reciprocal Rank (MRR): |𝑄| MRR = 1 |𝑄| ∑ 1 rank𝑖 𝑖=1 - Recall@k: Fraction of relevant documents retrieved in top-k - F1 Score: Harmonic mean of precision and recall for answer extraction - Hop-wise accuracy: Accuracy stratified by reasoning depth (1-hop, 2-hop, 3-hop) Results and Discussion. Table 1 and Figure 1 present the comparative performance of three retrieval strategies. Table 1. Comparative Performance of Retrieval Strategies Metric Vector Graph GNN Accuracy@5 (%) 38.6 38.6 48.6 MRR 0.240 0.246 0.323 Recall@5 0.386 0.386 0.486 F1 Score 0.386 0.386 0.486 Avg. Latency (ms) 0.1 0.4 0.1 Index Time (s) 0.01 0.01 0.04 The GNN-based approach delivers the strongest performance across all metrics, achieving a 10% accuracy gain over the baseline. It shows particular strength in MRR (0.323 vs 0.240), indicating that relevant documents tend to rank higher when this method is used. Notably, the vector and graph-enhanced approaches yield comparable accuracy results, suggesting that simple cooccurrence-based graph expansion offers limited benefit on its own without more sophisticated embedding techniques. Multi-hop Reasoning Performance. Figure 2 illustrates performance stratified by reasoning depth. Notably, the GNN approach shows different strengths across query complexity levels: - 1-hop queries: Vector (22.5%), Graph (22.5%), GNN (40.0%) - 2-hop queries: Vector (73.3%), Graph (66.7%), GNN (33.3%) “Fan va texnologiyalar taraqqiyoti” Ilmiy-texnikaviy jurnal №2/2026 216 Computer science and information-communication systems - 3-hop queries: Vector (46.7%), Graph (53.3%), GNN (86.7%) Fig. 1 Overall performance comparison across retrieval strategies The results uncover an interesting trend: the GNN approach excels on both simple single-hop and complex three-hop queries, yet shows a relative dip in performance on two-hop queries. This points to GNN embeddings being well-suited for capturing direct relationships and intricate multihop patterns, while potentially needing further tuning to handle intermediate complexity. The vector baseline, on the other hand, performs surprisingly well on two-hop queries — likely a reflection of how the synthetic dataset was constructed, where two-hop relationships are more explicitly represented in the document text. On three-hop queries — the most complex reasoning category — the GNN approach achieves 86.7% accuracy, substantially outperforming both baselines and underscoring its strength in capturing long-range dependencies through graph structure. Fig. 2 Performance by query complexity (multi-hop reasoning) Ablation studies were carried out to better understand the contribution of each individual component: Development of science and technology Scientific and technical journal №2/2026 217 Informatika va axborot-kommunikatsion tizimlar Graph expansion hops: 1-hop: 36.2% accuracy; 2-hop: 38.6% accuracy (used in experiments); 3hop: 35.1% accuracy (introduces noise) The optimal performance at 2-hop expansion suggests a balance between capturing relevant relationships and avoiding noise accumulation. Beyond 2 hops, the graph expansion includes too many peripherally related entities that dilute the relevance signal. GNN architecture: - 1-layer GCN: 42.3% accuracy - 2-layer GCN: 48.6% accuracy (used in experiments) - Random node features (no GNN): 38.9% accuracy The 2-layer architecture demonstrates clear advantages over both shallower networks and random features, confirming that the GCN effectively learns meaningful graph structure. Deeper networks were not tested, however, given the risk of over-smoothing that arises in smaller graphs. Testing various weighting schemes for the fusion parameter in the graph-enhanced approach revealed that relying solely on graph scoring produced only 31.2% accuracy, while peak performance was achieved at the optimal balance point — highlighting that preserving semantic vector similarity remains essential even when graph structure is incorporated. Computational Complexity Method Index Time (s) Query Time (ms) Vector 0.010 0.1 Graph 0.012 0.4 GNN 0.041 0.1 The computational analysis surfaces some noteworthy trade-offs. Although GNN demands roughly 4 times longer indexing time due to its two-layer graph convolution process, query-time performance remains on par with the vector baseline at 0.1ms. This efficiency is possible because GNN pre-computes node embeddings during indexing, allowing for fast similarity search at query time. Graph-enhanced retrieval carries the highest query latency at 0.4ms, a result of the runtime graph traversal operations performed per query. That said, all three methods maintain submillisecond query latency, keeping them well within the range of real-time applications. The upfront indexing cost is spread across many queries over time, making the GNN approach especially appealing in scenarios where query volume is high relative to the frequency of index updates. The experiments conducted on synthetic data yield several key insights into graph-enhanced RAG systems. GNN-based retrieval delivers a 10% absolute improvement over the vector baseline (48.6% vs 38.6%), while simple graph expansion without learned embeddings contributes minimal gains. GNNs prove particularly effective at handling both straightforward direct relationships and complex multi-hop patterns, and query latency stays practical across all approaches at under 1ms. Additionally, the quality of entity extraction has a significant bearing on the performance of graphbased methods. Surprising Observations: One unexpected finding was the near-identical performance of the vector and graph-enhanced approaches, both sitting at 38.6%. Closer analysis pointed to noisy connections introduced by the simple co-occurrence-based graph construction and entity extraction process, which ended up diluting relevance signals — underscoring just how critical high-quality knowledge graph construction is for effective graph-enhanced retrieval. The non-monotonic performance pattern observed across hop complexity levels (GNN: 40%→33%→87%) also raises the possibility that synthetic data may not fully reflect real-world query distributions. Meanwhile, the vector baseline's strong showing on two-hop queries points to considerable overlap between document content and two-hop relationships within the synthetic dataset. Limitations: Several limitations are worth acknowledging: the synthetic data may not fully mirror the complexity and query distributions found in real-world settings; the simple string-matching “Fan va texnologiyalar taraqqiyoti” Ilmiy-texnikaviy jurnal №2/2026 218 Computer science and information-communication systems approach to entity extraction restricts how effectively the graph can be utilized; the small dataset size — 200 documents and 70 queries — prevents meaningful statistical significance testing; TFIDF embeddings fall short of the representational power offered by transformer-based alternatives; and the evaluation relies on exact string matching rather than accounting for semantic equivalence. Conclusion. This study demonstrates that knowledge graph integration can meaningfully improve RAG retrieval — but only when graph structure is encoded through learned GNN representations. On synthetic data, GNN-based retrieval surpasses the vector baseline by 10 percentage points and holds a clear edge on complex multi-hop queries, all while maintaining submillisecond latency. Naïve graph expansion, by contrast, offers little added value and can introduce noise when the knowledge graph and entity extraction are of poor quality. Going forward, priority should be placed on robust entity linking and higher-quality knowledge graph construction, the adoption of stronger transformer-based embeddings, and validation of graph-enhanced retrieval on large-scale real-world datasets with more diverse query types — ideally extending evaluation to cover end-to-end generation quality alongside retrieval accuracy