Objectives/Goals: This is a scoping review of the use of generative AI (GenAI) for qualitative analysis in the health sciences. The primary objectives are to summarize the methods used for qualitative analysis based on the approach (e.g., inductive vs. deductive), metrics for examining GenAI’s performance by approach type, and best practices for improving performance. Methods/Study Population: We searched six databases (PubMed, EMBASE, CINAHL, Scopus, Web of Science, and PsycINFO) using a comprehensive search string tailored to each database. We identified 5,853 unique results, of which 223 were identified as potentially relevant after review of abstracts. We will conduct a full manuscript review for each potentially relevant result to confirm inclusion, including use of GenAI for qualitative analysis of physical or mental health text data where a full-length paper is available. Two study team members will review each manuscript to confirm inclusion prior to data extraction and coding. We will also enter each manuscript into Yale Clarity, a secure GenAI platform powered by multiple large language models (e.g., ChatGPT, Claude), to examine GenAI’s capacity to facilitate manuscript screening and coding. Results/Anticipated Results: We will code included articles for: the research topic area; type of qualitative data analyzed (interview, focus group, medical chart, text-based digital or social media content); type of study (original or secondary data, review); type of qualitative analysis (thematic analysis, content analysis, grounded theory); type of approach (inductive, deductive); steps in analytic process that generative AI was used (initial open coding and sense making, final coding based on established codebook, identification of overarching themes and narratives); GenAI platform(s) and large language model(s) used; types of GenAI prompts used to facilitate analyses; processes and measures/metrics to evaluate the accuracy and quality of GenAI’s results; outcomes and methods used to increase GenAI’s accuracy and/or quality. Discussion/Significance of Impact: Qualitative research is essential to understand patient perspectives and increase treatment effectiveness and accessibility for all. To actualize GenAI’s potential to facilitate rapid and large-scale qualitative analysis, we first need to understand its strengths, weaknesses, and best practices for maximizing GenAI’s accuracy and quality.
Ouellette et al. (Wed,) studied this question.