Key points are not available for this paper at this time.
Importance: Patient-centered research is essential for bridging the gap between research and patient care, yet patient perspectives are often inadequately represented in health research. Objective: To leverage artificial intelligence (AI) and natural language processing (NLP) to analyze a large dataset of patient messages, defining patient concerns and generating relevant research topics, and to quantify the quality of these AI-generated topics. Design, Setting, and Participants: This case series was conducted using an automated framework involving a 2-staged unsupervised NLP topic model and AI-generated research topic suggestions. The study was based on deidentified patient portal message data from individuals with breast or skin cancer at Stanford Health Care and 22 affiliated centers over July 2013 to April 2024. Exposures: A widely used large language model (ChatGPT-4o OpenAI; April 2024) was used and guided through multiple prompt-engineering strategies to perform multilevel tasks, including knowledge interpretation and summarization (eg, interpreting and summarizing the NLP-defined topics), knowledge generation (eg, generating research ideas corresponding to patients' issues), self-reflection and correction (eg, ensuring and revising the research ideas after searching for scientific articles), and self-reassurance (eg, confirming and finalizing the research ideas). Main Outcomes and Measures: Three breast oncologists (J.L.C., A.W.K., F.R) and 3 dermatologists (K.Y.S, J.Y.T., E.L.) evaluated the meaningfulness and novelty of the AI-generated research topics using a 5-point Likert scale (1 representing exceptional to 5 representing poor). Mean (SD) scores for meaningfulness and novelty were computed for each topic. Results: A total of 614 464 patient messages were analyzed from 25 549 individuals, 10 665 with breast cancer (98.6% female) and 14 884 had skin cancer (49.0% female). The overall mean (SD) scores for meaningfulness and novelty were 3.00 (0.50) and 3.29 (0.74), respectively, for breast cancer topics and 2.67 (0.45) and 3.09 (0.68), respectively, for skin cancer topics. One-third of the AI-suggested research topics were highly meaningful and novel when both scores were lower than the average (5 of 15 for breast cancer and 6 of 15 for skin cancer). Notably, two-thirds of the AI-suggested topics were novel (10 of 15 for breast cancer and 11 of 15 for skin cancer). Conclusions and Relevance: This case series demonstrates that AI/NLP-driven analysis of large volumes of patient messages can generate quality research topics in cancer care that reflect patient perspectives, providing valuable guidance for future patient-centered health research endeavors.
Kim et al. (Thu,) studied this question.