Beyond Keywords: Comparing LLM-Assisted Topic Modeling Approaches Across Political Communication Contexts

This study provides a comprehensive evaluation of large language model (LLM)-assisted topic modeling across diverse political communication datasets, proposing a conceptual distinction between topic categorization (manifest subject mapping) and concept induction (latent construct discovery). We compare four methods, LDA, BERTopic, TopicGPT, and LLooM paired with four proprietary LLMs and one open-weights model across social media posts, party platforms, and legislative bills. Using a multi-dimensional framework of alignment, coherence, diversity, and human interpretability, we find that TopicGPT excels at high-precision topic categorization, achieving peak semantic alignment especially in structured policy contexts. Conversely, LLooM functions as a powerful tool for concept induction; although it may diverge from manifest ground-truth labels, it frequently demonstrates stronger text-topic relevance by capturing latent discursive logics that deductive approaches may overlook. Our findings further reveal that GPT-family models generally mitigate semantic vagueness, whereas the open-weights model shows a distinctive capacity for thematic diversity and performs competitively in some alignment settings, though its performance varies across contexts. Human annotation uncovers a semantic compression loss, where models identify rich latent themes but struggle to condense them into concise labels, highlighting the persistent need for human interpretive oversight. We conclude by offering an epistemological and practical framework to guide communication scholars in selecting GenAI-assisted methods that align with their theoretical level of meaning.