Effective Summarization of ChatGPT User Feedback: Integrating Topic Detection with Markov Chains

Abstract In the era of ChatGPT, user feedback is essential for understanding this advanced technology and navigating the evolving field of artificial intelligence. To gain insights into the risks, limitations, and areas for improvement, our study focuses on analyzing Twitter interactions. We address the unique challenges posed by Twitters characteristicsbrevity, informality, and rapid temporal changesin understanding and summarizing user-generated content related to ChatGPT. Our research emphasizes interest-based multi-tweet topic detection and summarization, tackling challenges such as brevity, noise, and dynamic content. To address these challenges, we introduce a approach that merges Latent Semantic Analysis (LSA) and Non-negative Matrix Factorization (NMF) for effective topic detection in 500,036 English tweets from the first three months after ChatGPT’s announcement. Our method is rigorously compared with traditional models such as Latent Dirichlet Allocation (LDA) and BERTopic. The coherence scores, which reflect the quality of the extracted topics, show values of 0.68 for LDA, 0.51 for BERTopic, and an impressive 0.92 for our proposed model, demonstrating a 92.67 % improvement in effectiveness. Our goals extend beyond topic detection to include comprehensive topic summarization using Markov Decision Processes (MDP). When compared to LexRank, TextRank, and TopicRank, our method significantly enhances multi-tweet summarization accuracy, as evidenced by improved ROUGE evaluation metrics. In conclusion, our approach represents a significant advancement in the accuracy of automatic multi-tweet summarization.