Background. Pandemic preparedness in low- and middle-income countries (LMICs) is hindered by limited access to complex disease surveillance guidelines, particularly among frontline health workers with suboptimal English proficiency. Uganda's Integrated Disease Surveillance and Response (IDSR) guidelines, while comprehensive, remain largely underutilized due to their length, English-only format, and frequent health worker turnovers. Methods. We developed HEAL, a custom bilingual chatbot leveraging GPT-4-turbo and retrieval-augmented generation (RAG) to make Uganda's IDSR guidelines conversationally accessible in English and Luganda. We trained an MBart model on over 40,000 parallel sentence pairs for translation. Six disease surveillance experts from Uganda's Ministry of Health evaluated HEAL against three commercial large language models (GPT-3.5, GPT-4, and Gemini) using 50 field-generated questions across four metrics: relevance, coverage, coherence, and harm. Results. HEAL achieved higher median scores than commercial LLMs for three of six reviewers, with highly significant differences observed (p<0.001). HEAL significantly outperformed other models in coverage and relevance. Qualitative feedback from frontline health workers highlighted HEAL's utility in reducing false alerts and improving resource allocation, though infrastructural challenges (internet connectivity, device availability) were noted as barriers. Conclusions. This study demonstrates the feasibility of deploying a bilingual, RAG-based conversational AI chatbot for disease surveillance in resource-limited settings. HEAL represents the first integration of a Ugandan indigenous language into an LLM-based health tool, addressing linguistic inequity in AI applications and providing a scalable framework for LMICs to leverage generative AI for pandemic preparedness.
Atwine et al. (Mon,) studied this question.