Are AI chatbots reliable sources for parents to ask about their concerns about childhood vaccination?

Background: The increasing use of AI-powered chatbots for health-related inquiries has positioned them as potential tools in combating vaccine hesitancy among parents. However, the reliability of these tools in delivering accurate, consistent, and actionable information on childhood immunization remains underexplored. Objective: This study aimed to assess how the most widely used AI chatbots guide parents seeking information about childhood vaccines, rather than comparing them against each other. Methods: A cross-sectional comparative design was used. Three freely accessible, commonly used AI conversational agents were each presented with 9 frequently asked parental questions about childhood vaccinations. To maintain neutrality and avoid brand-based interpretation bias, these chatbots are anonymized in the study as AICB 1, AICB 2, and AICB 3. Responses were independently evaluated across four domains—accuracy, consistency, information sufficiency, and source reliability—using a 5-point Likert scale. Each chatbot was tested in two temporally distinct sessions to assess consistency. Results: All chatbots generated scientifically accurate and temporally consistent responses. Mean composite scores were highest for AICB 1 (4.9), followed by AICB 2 (4.7) and AICB 3 (4.1). The performance difference was statistically significant (H = 11.27, p < 0.01). Despite the statistically significant differences between the agents, all three chatbots achieved high scores across the four evaluated dimensions. Conclusion: We need to know that AI chatbots can offer accessible, generally reliable information on vaccines and provide more reliable, accurate data than profit-driven websites; they should be used as supplementary tools, especially when addressing sensitive public health topics like childhood immunization.