Structure-Aware Graph-RAG for Small Language Models: Reducing Hallucinations and Improving Multi-Step Reasoning in Specialized Domains

Retrieval-augmented generation (RAG) has been widely used to improve factual grounding in small language models (SLMs). However, many existing retrieval methods rely mainly on shallow semantic similarity between queries and text chunks. As a result, the retrieved evidence is often fragmented or weakly connected, which can limit multi-hop reasoning and sometimes lead to hallucinated answers. In this work, we propose Hallucination-Aware Multi-Objective Ant Colony Optimization (HA-MOACO), a structure-aware Graph-RAG framework that models relationships between pieces of evidence and searches for useful reasoning paths in a graph structure. The framework first constructs a domain knowledge graph and identifies several candidate entry nodes related to the query. Starting from these nodes, an ant colony optimization (ACO) strategy is used to explore possible evidence paths. During this process, multiple signals are considered simultaneously, including semantic relevance, structural connectivity between nodes, and reliability indicators that help reduce contradictory or low-confidence evidence. The selected evidence paths are then used to build a compact and grounded context for the language model. This process helps keep important reasoning connections while reducing the influence of irrelevant information. Experimental results on reasoning-focused benchmarks show that the proposed framework improves answer accuracy by about 6%, increases F1 by roughly 5–6%, and improves exact-match scores by around 7% compared with strong graph-based retrieval baselines. At the same time, the hallucination rate is reduced by about 40%, while retrieval latency is roughly 20% lower. These results suggest that combining structure-aware retrieval with reliability-oriented optimization can improve both reasoning quality and factual consistency in SLMs, while still remaining efficient for practical deployment.