Key result
ChatGPT-3.5 provides fair knee osteoarthritis answers with a 51 DISCERN score but lacks reliable citations.
Why the study?
Patient education is crucial in healthcare, and with the adoption of artificial intelligence, the use of online chatbots is inevitable, warranting evaluation of ChatGPT-3.5 in answering common patient questions about knee osteoarthritis.
Cross-Sectional (n=10)
Answers are grounded in this paper and cite their sources.
ChatGPT-3.5 provides fair but unreferenced responses to knee osteoarthritis questions at a college reading level, indicating it should only supplement, not replace, physician-led patient education.
Background Patient education is an important component in providing high quality healthcare, especially in the context of orthopedic surgery. In the current era of continuous technological advancements and the adoption of artificial intelligence in healthcare, the use of online chatbots in patient education is inevitable. The purpose of this paper is to evaluate ChatGPT-3.5’s effectiveness in answering common patient questions about knee osteoarthritis. Methods Ten frequently asked questions were collected from ten separate healthcare institution pages and input into ChatGPT-3.5. The questions were then analyzed for reliability and completeness using the DISCERN instrument and the Journal of the American Medical Association (JAMA) Benchmark criteria. The readability was analyzed using the Flesch Kincaid scoring system. Results Of the ten questions, the average DISCERN score was 51. Three responses were considered good, six were fair, and one was poor. The JAMA Benchmark criteria was zero for all responses. The average Flesch Kincaid grade level score was 29.33, indicating a college grade reading level. Conclusion ChatGPT-3.5 may have the potential to be an informative tool for patients with questions about knee osteoarthritis. It was able to provide fair responses, however, some inquiries required clarification and all responses lacked reliable citations. Furthermore, the responses were written at a college grade reading level, which limits its utility. Therefore, proper patient education should be conducted by orthopedic surgeons. This highlights the need for patient education resources that are both accessible and comprehensible.
No takes yet. Share an insight, caveat, or question.
Gilmore et al. (2024) conducted a cross-sectional in Knee Osteoarthritis (n=10). ChatGPT-3.5 was evaluated on Average DISCERN score. ChatGPT-3.5 provided fair responses to common patient questions about knee osteoarthritis with an average DISCERN score of 51, though all responses lacked reliable citations and were written at a college reading level.
Synapse has enriched 10 related papers. Consider them for comparative context: