Deciphering Public Voices in the Digital Era

Problem, research strategy, and findings Planners are increasingly using online public engagement approaches to broaden their reach in communities. This results in substantial volumes of digital, text-based public feedback data, making it difficult to analyze efficiently and derive meaningful insights. We explored the use of the novel large language model (LLM), ChatGPT, in analyzing a public feedback data set collected via online submissions in Hamilton City (New Zealand) in response to a proposed local plan change. Specifically, we initially employed zero-shot prompts with ChatGPT for tasks like summarizing, topic identification, and sentiment analysis and compared the results with those obtained by human planners and two standard natural language processing (NLP) techniques: latent Dirichlet allocation (LDA) topic modeling and lexicon-based sentiment analysis. The findings show that zero-shot prompting effectively identified political stances (accuracy: 81.7%), reasons (87.3%), decisions sought (85.8%), and associated sentiments (94.1%). Although subject to several limitations, ChatGPT demonstrates promise in automating the analysis of public feedback, offering substantial time and cost savings. In addition, few-shot prompting enhanced performance in more complex tasks, such as topic identification involving planning jargon. We also provide insights for urban planners to better harness the power of ChatGPT to analyze citizen feedback.