ChatGPT-4 Generates More Accurate and Complete Responses to Common Patient Questions About Anterior Cruciate Ligament Reconstruction Than Google’s Search Engine

PurposeThe purpose of this study was to replicate a patient's internet search to evaluate ChatGPT's appropriateness in answering common patient questions about anterior cruciate ligament reconstruction (ACLR) compared to a Google web search.MethodsA Google web search was performed by searching the term 'anterior cruciate ligament reconstruction'. The top 20 frequently asked questions (FAQs) and responses were recorded. The prompt "what are the 20 most popular patient questions related to 'anterior cruciate ligament reconstruction'?" was inputted into ChatGPT and questions and responses were recorded. Questions were classified based on the Rothwell system and responses assessed via Flesch-Kincaid Grade Level (FKGL) , correctness, and completeness were for both Google web search and ChatGPT.ResultsThree of twenty (15%) questions were similar between Google web search and ChatGPT.The most common question types amongst the Google web search were value (8/20, 40%), fact (7/20, 35%) and policy (5/20, 25%). The most common question types amongst the ChatGPT search were fact (12/20, 60%), policy (6/20, 30%), and value (2/20, 10%). Mean FKGL for Google web search responses was significantly lower (11.8 ± 3.8 vs. 14.3 ± 2.2; P=0.003) than for ChatGPT responses. The mean correctness for Google web search question answers was 1.47 ± 0.5 and mean completeness was 1.36 ± 0.5. Mean correctness for ChatGPT answers was 1.8 ± 0.4 and mean completeness was 1.9 ± 0.3 which were both significantly higher than Google web search answers (P= 0.03 and P = 0.0003).ConclusionsChatGPT-4 generated more accurate and complete responses to common patient questions about ACLR than Google's search engine The purpose of this study was to replicate a patient's internet search to evaluate ChatGPT's appropriateness in answering common patient questions about anterior cruciate ligament reconstruction (ACLR) compared to a Google web search. A Google web search was performed by searching the term 'anterior cruciate ligament reconstruction'. The top 20 frequently asked questions (FAQs) and responses were recorded. The prompt "what are the 20 most popular patient questions related to 'anterior cruciate ligament reconstruction'?" was inputted into ChatGPT and questions and responses were recorded. Questions were classified based on the Rothwell system and responses assessed via Flesch-Kincaid Grade Level (FKGL) , correctness, and completeness were for both Google web search and ChatGPT. Three of twenty (15%) questions were similar between Google web search and ChatGPT.The most common question types amongst the Google web search were value (8/20, 40%), fact (7/20, 35%) and policy (5/20, 25%). The most common question types amongst the ChatGPT search were fact (12/20, 60%), policy (6/20, 30%), and value (2/20, 10%). Mean FKGL for Google web search responses was significantly lower (11.8 ± 3.8 vs. 14.3 ± 2.2; P=0.003) than for ChatGPT responses. The mean correctness for Google web search question answers was 1.47 ± 0.5 and mean completeness was 1.36 ± 0.5. Mean correctness for ChatGPT answers was 1.8 ± 0.4 and mean completeness was 1.9 ± 0.3 which were both significantly higher than Google web search answers (P= 0.03 and P = 0.0003). ChatGPT-4 generated more accurate and complete responses to common patient questions about ACLR than Google's search engine