An AI-powered RAG Conversational Agent for Enhancing Patient Recall of Surgical Consultations: A Proof-of-concept Study

Abstract Background Surgical consultations contain dense information that patients struggle to recall, and existing tools do not provide access to individualized clinical dialogue. Retrieval-Augmented Generation (RAG) can ground language-model responses in source transcripts, enabling patients to revisit the specific advice given during their consultation. Objectives This study evaluates the technical feasibility and performance of a RAG conversational agent built on synthetic plastic surgery encounters. Methods Twenty simulated patient cases were generated across ten plastic surgery procedures, each containing four sequential visits. Transcripts were converted to audio and transcribed using Whisper to simulate real-world ambient capture. The transcriptions were ingested into a Vertex AI RAG system powered by Gemini 2.0 Pro. Two hundred patient-style questions were developed across domains including procedural details, risks, recovery, medications, and clinical parameters. Responses were evaluated against gold-standard transcript answers for accuracy, precision, recall, and error type. Readability was assessed using standard metrics. Results The system achieved 99.0% accuracy (198/200), with perfect precision and 0.99 recall. One question demonstrated appropriate uncertainty handling when terminology differed from transcript content. No hallucinations or incorrect substitutions occurred. Readability analysis showed a mean Flesch-Kincaid Grade Level of 8.56 and a Reading Ease score of 60.87, indicating accessible patient-level language. Conclusions A RAG conversational agent grounded in consultation transcripts can deliver highly accurate, patient-friendly recall support, with strong factual reliability and appropriate uncertainty handling. Temporal integration remains the primary area for improvement. These findings demonstrate the feasibility of transcript-based patient assistance and support future development toward real-world deployment.