Aims: Many patients seek accurate, understandable information about their disease and treatment, turning to the internet or messaging providers. This study aims to validate chatbots' ability to deliver accurate information, contributing to the literature on AI's role in cancer care and helping to improve these tools for patients and caregivers. Methods: A set of questions about hematologic malignancies was created with input from oncologists and reputable websites and then submitted to ChatGPT 3.5. Each response was rated by hematology-oncology physicians from strongly disagree (1) to strongly agree (5) regarding its accuracy and usefulness for patients, with multiple reviewers ensuring consistency. Results: The general queries category received a higher average score of 3.38 compared to 3.06 in the novel therapies category, indicating relatively better satisfaction. However, no question achieved scores greater than 4 (agree) or 5 (strongly agree), with most scores ranging from 3.0 to 3.8, reflecting a neutral stance, suggesting room for improvement. Conclusions: ChatGPT struggled with providing current and specific information for patient-specific queries and novel therapies, especially in rapidly advancing fields like acute myeloid leukemia. These deficiencies are likely due to AI's reliance on large data sets, leading to less influence from novel therapies.
Nong et al. (Wed,) studied this question.