AarogyaSathi: Design and Preliminary Evaluation of a Multilingual Public Health Information Chatbot

AarogyaSathi is a public health information chatbot that our team built in response to Smart India Hackathon problem statement SIH25049. It brings together a curated knowledge base, multilingual retrieval, language model answers, claim verification, emergency phrase rules, and an experimental detector that tracks symptom reports by pincode. Here we describe the version running on Oracle Cloud and ask how far the available tests support it. We read the source code, inspected the live service, reran the repository’s component benchmarks, and sent the system synthetic conversations. All 37 automated tests in the 26 October run passed, and the supplied evaluation sets gave 30/30 correct intent labels, 15/15 expected retrieval targets, and 34/34 expected claim verdicts. Training overlap and configuration differences limit the interpretation of these scores. Of the 30 intent questions, 14 also appear in the training data, and the benchmark thresholds differ from the production settings. The conversations turned up problems the benchmarks missed. A sentence saying the user had no chest pain triggered an emergency response, and an Odia question about prevention got no useful answer. Reading the code revealed a further error, in how vaccination ages are converted. The web interface worked, and a subsequent Meta WhatsApp test-number demonstration confirmed replies for greeting, dengue-information, and claim-checking prompts. Integration with official outbreak data re-mained unverified. Our findings are preliminary engineering evidence about how the implemented response paths behave and where further validation is needed. Clinical effectiveness was not assessed.