Large language model-based online review classification for sub-feature-level customer opinion analysis

In recent years, many studies have analyzed online reviews to understand customer preferences and requirements for product features. However, most of them have focused on feature categories, whereas companies need to analyze customer preferences regarding sub-features to gain practical insights for product development. To bridge the gap, this study proposes a new method for sub-feature-level review analysis. First, text review sentences are embedded into vectors using a large language model. A sentence bidirectional encoder representation from transformer (SBERT) model is employed. Next, the method trains a neural network model that classifies reviews into sub-features. The input data are sentence vectors and the outputs are class labels indicating product sub-features. To address the problem of highly imbalanced labels in review data, a new loss function is proposed based on evaluation metrics. The proposed method was tested using smartphone and headphone reviews collected online. The results showed that the new method achieved higher performance, i.e., F1 scores over 0.80, than a previous BERT-based classifier (F1 scores between 0.39 and 0.69). In addition, the new loss function provides a more balanced precision/recall for all the classes. The developed approach will help companies extract customer opinions at the product sub-feature level and has practical implications for early-stage product design.