Background: Artificial Intelligence (AI) has advanced tremendously over the past few years and has demonstrated great potential in supporting existing healthcare systems. Due to their rapid data processing, AI tools are increasingly being used by people to assess health concerns and make preliminary decisions about seeking health care. This study aims to analyse the diagnostic accuracy of AI tools in determining common pediatric emergencies. Methods: 120 pediatric case reports were collected from open access, peer reviewed journals based on the inclusion criteria. The cases were rewritten in layman language and entered into three AI tools : ChatGPT, WebMD Symptom Checker and Mayo Clinic symptom checker and the top three differential diagnoses were analyzed. A standard 3-point scoring system was used to assess the diagnostic performance of each tool by two independent reviewers. Statistical analysis of the scores were done using Friedman’s test followed by post hoc pairwise comparisons with Bonferroni correction. Results: Analysis showed that ChatGPT had the highest average mean of 0.95 ± 0.94 and was ranked the highest with a mean rank of 2.42 out of the three tools studied. Friedman’s test and post hoc analysis with Bonferroni correction confirmed the statistical significance (p < .001). There was no statistical difference between the scores for WebMD and Mayo Clinic symptom checkers (adjusted p = 1.000). Conclusion: With increasing access and a large number of people resorting to AI as a reliable tool for diagnosis, ChatGPT seems to show potential in this regard. Although the diagnostic accuracy seems significant , it is important to consider the limitations of such tools which do not take into consideration the contextual information, social determinants of health and emotional aspects of clinical presentation.
S Natarajan (Thu,) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: