Zero or Hero? Exploring the Use of AI Tools for Thematic Analysis in Science Communication Evaluation

Practitioners in informal science learning institutions are often expected to analyse evaluation data with limited time and expertise, and some are turning to artificial intelligence (AI) tools for support. To examine what these tools can realistically offer, we used interview data from an outreach event. We compared single-prompt thematic analyses produced by five AI tools with each other and with an iterative practitioner–AI analysis. We also explored how qualitative researchers judged the feasibility, trustworthiness and practical value of AI support. The AI tools consistently identified core patterns, but their outputs were largely descriptive, varied across trials and included hallucinated quotes. We recommend a hybrid approach in which AI assists with organising data while researchers retain responsibility for interpretation.