Through Thick and Thin: Comparing Traditional Qualitative Analysis and Natural Language Processing Techniques Using Narrative Data from Police Officers

This study compared traditional (human-coded) qualitative analysis and (machine-coded) natural language processing (NLP) techniques on the same open-ended survey responses from police officers. Both strategies uncovered similar broad themes, but the traditional approach revealed more depth and nuance, including an additional category of themes. Therefore, NLP-basedapproaches cannot currently replace traditional qualitative analysis when depth and contextual richness are sought. The NLP analysis provided time savings and could detect whether survey items (e.g., officer ethnicity) were linked to certain themes. When rapid, high-level theme identification is the goal, or when analyzing large volumes of text data, NLP tools are efficient and reproducible options. The two coding approaches can potentially complement each other within the same study. For instance, NLP can identify broad patterns early in a project to guide deeper, subsequent traditional analysis.