Accuracy, Completeness, and Clarity of EAU Guidelines Bot for Renal Transplantation

Background/Objectives: Recently, the European Association of Urology (EAU) guidelines presented the EAU guidelines bot to assist urologists in the reading of the guidelines; however, to date, no external validation is available. The aim of our study is to assess the accuracy, completeness, and clarity of the guidelines bot in reading renal transplantation guidelines and clinical cases. Methods: A total of 103 questions derived from the recommendations included in the EAU guidelines on renal transplantation, together with 10 distinct clinical scenarios, were created. Each question and clinical case was submitted to the EAU guidelines bot, and the corresponding responses were independently evaluated by two expert urologists in terms of accuracy, completeness, and clarity. Each domain was scored using a 5-point Likert scale. In cases of disagreement between the two reviewers, a third urologist was consulted. Accuracy, completeness, and clarity were analyzed for the clinical cases and per chapter and grade of recommendation for the questions. Results: Overall, 103 questions were generated. Regarding accuracy, 101/103 responses (98%) were classified as accurate, corresponding to scores of 4–5, whereas 2/103 responses (2%) showed fair accuracy, corresponding to a score of 3. No responses were considered inaccurate, defined as a score of 1–2. In terms of completeness, 101/103 responses (98%) were judged to be complete, with scores of 4–5, while 2/103 responses (2%) were rated as fairly complete, with a score of 3. No responses were classified as incomplete. With regard to clarity, 102/103 responses (99%) were considered clear, corresponding to scores of 4–5, whereas 1/103 response (1%) was rated as fairly clear, with a score of 3. No responses were deemed unclear. No significant differences were observed when comparing responses related to strong versus weak recommendations. For the clinical scenarios, 8/10 responses (80%) were rated as accurate, with scores of 4–5, while 2/10 responses (20%) showed fair accuracy, with a score of 3. Regarding completeness, all 10 responses (100%) were classified as fairly complete, corresponding to a score of 3. Finally, in terms of clarity, 8/10 responses (80%) were judged to be clear, with scores of 4–5, whereas 2/10 responses (20%) were rated as fairly clear, with a score of 3. Conclusions: The EAU guidelines bot represents an accurate tool for renal transplantation. Besides some occasional inaccuracies the bot is an excellent tool for urologists, especially to clarify doubt about the EAU guidelines. At the same time, it can give good support when facing a clinical case.