Addressing hallucinations in generative AI agents using observability and dual memory knowledge graphs

Generative AI has rapidly progressed from chatbots and assistants to agents across diverse applications and domains. Generative AI agents demonstrate sophisticated operation through autonomy, tool use and decision making with minimal human input. Despite these performance gains, agents are still impacted by the foundational limitations of Generative AI models. Among these, hallucinations are a major limitation that affects agent operation in real-world settings, leading to risk and loss. Several recent work aim to address hallucinations through methods such as retrieval-augmented generation and reflection prompting, however, these only provide partial improvements. An effective yet underexplored approach is in the observability data generated by an agent in its deployed and operational settings. Drawing on agent observability data, this paper proposes a dual memory knowledge graph approach that integrates Semantic and Observability Memory to address hallucinations in Generative AI agents. Semantic Memory provides organized domain knowledge for precise factual grounding. Observability Memory transforms logs, traces, and execution results into agent validated planning histories. Hallucinations are then addressed by grounded planning in verified past interactions with known, reliable outcomes. This approach is evaluated in a two-stage experimental setup aligned with its dual memory design. Observability memory is evaluated on the HotpotQA dataset to assess its impact on reasoning grounding, using metrics that capture both factual accuracy and reasoning hallucinations. The SM3-Text-to-Query benchmark and Synthea-based medical QA datasets are used to assess factual grounding of the semantic memory. Results from both experiments demonstrate reductions in hallucinations, with semantic memory for contextual grounding reducing factual hallucinations, and observability memory for reasoning grounding reducing faithfulness hallucinations.