Reliable Cardiovascular Question Answering with Validated Knowledge Graphs and Multi-Model Embeddings (Preprint)

BACKGROUND Artificial Intelligence (AI)-powered medical question-answering systems significantly improve access to reliable healthcare information, yet challenges remain, particularly hallucinations in large language models (LLMs), where AI produces misleading or fabricated content. These risks are especially critical in clinical contexts. Ensuring accuracy and trustworthiness requires validated, domain-specific datasets and robust semantic alignment. OBJECTIVE This study aims to develop and evaluate an AI-driven question-answering system for cardiovascular diseases using a validated dataset, knowledge graph and advanced semantic models to improve the accuracy, interpretability and trustworthiness of responses. Each answer’s reliability is demonstrated and supported by the source of the information, a reliability rating based on the CRAAP method, relevant references and disease-related images to support transparency and user trust. METHODS We have curated a dataset from trusted medical sources (Mayo Clinic, Cleveland Clinic, NHS, British Heart Foundation, MedlinePlus), validated using the CRAAP method. The data is structured into a knowledge graph via an automated pipeline. Semantic matching is performed using PubMedBERT, BioBERT and Sentence Transformer, while a T5-based model is employed to rephrase low-confidence queries for improved alignment. RESULTS The system is evaluated using 100 queries across five categories: straightforward, complex (long queries), general spelling mistakes, misspelled disease names, and out-of-data queries. It demonstrates high accuracy for straightforward (99.64 ± 1.6%), complex (98.78 ± 2.68%), and general spelling mistake queries (99.1 ± 2.79%). Accuracy declines with misspelled disease names (54.56 ± 50.67%) due to limitations in correcting domain-specific errors. For out-of-data queries (11.99 ± 29.28%), the system refrains from speculative outputs and transparently indicates when no satisfactory answer is available. CONCLUSIONS This study demonstrates the successful development of a trustworthy and accurate AI-driven question-answering system for cardiovascular medicine. By integrating a validated knowledge graph, multi-model language embeddings and a T5-based paraphrasing model, the system delivers contextually appropriate responses supported by source ratings, references and medical images. While performance is strong across most query types, challenges with misspelled disease names and multi-intent queries remain. Future work will address these limitations through spelling correction, query segmentation and expansion to additional disease domains.