Standard retrieval-augmented generation works well when the knowledge you need lives in a flat document store. When it does not, when the relevant facts are distributed across a hierarchy of parent-child relationships and lateral relational edges, the performance degrades in ways that are both predictable and underexplored. This paper (arXiv:2607.14095) proposes HG-RAG, a framework that replaces cosine-similarity retrieval over dense node embeddings with directed graph traversal over a structured knowledge graph. The core claim is straightforward: if your knowledge is hierarchical, your retrieval should be too.
What HG-RAG Actually Does
The pipeline has three stages. First, a single LLM pass extracts a named entity anchor from the incoming query. Second, the system performs a structured traversal from that anchor: upward through parent nodes to capture containment context, laterally through relational neighbors (borders, trade partners, hostile relations), and downward through child nodes when the query warrants it. Third, the resulting subgraph, capped at 15 nodes, is serialised and passed to the answering model as context.
This is conceptually related to Microsoft's GraphRAG, which also builds community-structured graphs from source documents. The distinction is that HG-RAG explicitly distinguishes between structural edges (a contains relationship between country and city) and relational edges (a trade_with or hostile relationship between two cities). The traversal is directional and typed, not just a k-hop neighbourhood expansion over an undifferentiated graph. That specificity matters for hierarchical domains where the direction of an edge carries semantic weight.
The evaluation uses a synthetically generated geopolitical world graph at three scales: small (18 nodes), medium, and large (800 nodes). Four query types are tested: local fact, hierarchical, neighborhood, and multi-hop. The baseline is a dense retrieval system using cosine similarity over node embeddings, paired with the same Mistral 7B model used for answering. Grading is also done by Mistral 7B, which the paper correctly flags as a methodological concern.
Key Results
The headline numbers are striking. At large scale, the dense retrieval baseline achieves a factual accuracy of 0.022. HG-RAG achieves 1.857 on the same graphs. That is not a marginal improvement; it is a qualitative difference in whether the system is functional at all.
The multi-hop results are the most theoretically interesting. The baseline degrades monotonically as world size increases (LLM judge scores of 3.450, 2.817, 1.660 across small, medium, large). HG-RAG does the opposite: it improves with scale (3.233, 3.883, 4.100). The explanation offered is plausible. In small worlds, the 15-node subgraph cap captures a large fraction of the entire graph, introducing noise rather than signal. In large worlds, the same cap enforces meaningful filtering, and the traversal is doing exactly the work it was designed for: pulling a tight, relevant neighbourhood from a vast space of competing nodes.
- HG-RAG outperforms the flat baseline on hierarchical, neighborhood, and multi-hop query types across all world scales
- Baseline factual accuracy collapses to near-zero at 800 nodes; HG-RAG remains functional
- Multi-hop performance for HG-RAG improves with graph complexity, inverting the baseline trend
- Local fact queries show the smallest gap, consistent with the expectation that flat retrieval handles atomic lookups reasonably well
Methodological Concerns Worth Flagging
The paper is admirably self-critical about its limitations, which is worth acknowledging. The most significant issue is the evaluation pipeline. Using the same Mistral 7B instance as both the answering model and the judge introduces a self-consistency bias. The paper notes a concrete example of this: the judge awarded a score of 5 to two factually incorrect responses, apparently because they were formatted as detailed bullet points. This is a known failure mode of LLM-as-judge evaluation and is not unique to this work, but it does mean the absolute scores should be treated with caution. The relative ordering between HG-RAG and the baseline is likely more reliable than the absolute values.
The fixed 15-node subgraph cap is another limitation that the paper correctly identifies. A static cap is a reasonable starting point, but it creates a tension: too large relative to the graph and you get noise; too small relative to a complex multi-hop query and you miss relevant nodes. Adaptive capping based on query type or graph density would be a natural extension.
The named entity dependency is a real constraint on generality. The entire retrieval pipeline starts with entity extraction, so attribute-based queries without a clear named entity anchor fall outside the system's scope entirely. The paper frames this as a deliberate design choice to keep the core contribution legible, which is fair, but it does limit the range of real-world query types the system can handle without modification.
Finally, the evaluation is conducted on synthetic graphs. Geopolitical world graphs are a reasonable proxy for hierarchical knowledge, but they are clean and well-structured by construction. Real knowledge graphs carry noise, missing edges, inconsistent typing, and ambiguous entity boundaries. How HG-RAG degrades under those conditions is an open question.
Broader Implications
The core finding here reinforces something that the knowledge graph question answering literature has argued for some time: semantic similarity is not a substitute for structural reasoning. Dense retrieval works by finding nodes that look like the query. Graph traversal works by finding nodes that are structurally connected to the query anchor. For hierarchical domains, those are different sets, and the difference grows with graph size.
The scaling behaviour of multi-hop performance is the result most worth paying attention to. If HG-RAG's advantage over flat retrieval increases as graphs grow, that has direct implications for enterprise knowledge management, biomedical ontologies, and any domain where knowledge is naturally tree-structured and the graphs are large. The argument for hierarchy-aware retrieval gets stronger, not weaker, as the knowledge base scales.
This is a single-author paper run on consumer hardware with a 7B parameter model, which is both a limitation and a point in its favour for reproducibility. The code is available at github.com/Pranubot/HG-RAG. The methodology is clear enough to replicate, and the honest treatment of limitations makes it a useful reference point for anyone building retrieval systems over structured knowledge. The next steps, adaptive subgraph sizing, a stronger judge model, and evaluation on real-world noisy graphs, are obvious, but that is the nature of early-stage work that has something genuine to say.
Full paper: HG-RAG: Hierarchy-Guided Retrieval-Augmented Generation for Structured Knowledge Graphs (arXiv:2607.14095)