An Integrated Framework for Automated Web Scraping and Sentiment Analysis of Product Reviews

In the digital age, online product reviews play a pivotal role in influencing consumer choices.This paper presents an innovative tool that combines state-of-the-art natural language processing and sentiment analysis technique with ChatGPT technology.The tool enables automated web scraping and comprehensive sentiment analysis of product reviews, with a focus on aspect-based sentiment analysis and document-level sentiment aggregation.Our tool employs ChatGPT, a powerful language model, to conduct aspect-based sentiment analysis, extracting nuanced sentiments related to specific product attributes.This provides a granular understanding of consumer opinions.The tool also integrates advanced models for document-level sentiment analysis, enabling the aggregation of overall sentiment scores from multiple reviews.The framework comprises three main components: Web Scraping Module: this automates data collection from two major e-commerce platforms ensuring systematic extraction of reviews and metadata.Aspect-Based Sentiment Analysis Module:n leveraging ChatGPT, it categorizes opinions as positive, negative, or neutral and identifies specific product attributes.Document-Level Sentiment Analysis Module: this component provides an overall sentiment rating for the product.The tool can find applications in business, research, and consumer decision-making, allowing real-time product performance monitoring, market studies, and informed purchase decisions.Our integrated framework offers a robust solution for understanding and responding to consumer sentiments in online product reviews.By combining aspect-based and documentlevel sentiment analysis with ChatGPT's capabilities, it facilitates comprehensive insights into the digital marketplace.