ChatGPT has gained popularity for its user-friendly applications in medicine, including otolaryngology. However, concerns have surfaced about accuracy of information, especially in languages other than English. This study assessed ChatGPT's proficiency in answering questions in English and French about benign paroxysmal positional vertigo (BPPV), a common ENT presentation. We aim to address concerns regarding its performance with medical queries in a language beyond its primary training set. ChatGPT 3.5 was tested with questions taken from the guidelines of the American Academy of Otolaryngology-Head and Neck Surgery Foundation (AAO- HNSF), in English and French. Three users submitted 14 questions twice across distinct sessions, with different user-names and IP addresses. A two-stage translation process was used to ensure linguistic accuracy. A total of 14 questions related to BPPV were presented to ChatGPT. Two independent otolaryngologists, proficient in both English and French, reviewed and graded the responses on a 4-point Likert scale. Responses to all 14 questions were uniform in both English and French, demonstrating high consistency and native language level. Twelve of the 14 questions were comprehensively aligned with established guidelines (grade 4/4). The 2 other responses, although accurate, lacked detail on the risk of falls in older persons with BPPV (grade 3/4). ChatGPT can provide patients with detailed guideline-compliant responses to their medical queries in French or English. Using ChatGPT in both official Canadian languages could enhance patients’ access to medical information, as a complement to professional otolaryngologic expertise.
Magri et al. (Wed,) studied this question.