A hybrid transformer architecture for unsupervised aspect‐based sentiment analysis

Abstract Aspect‐based sentiment analysis focuses on discovering the sentiment polarity of specific aspects within textual data. The subtle complexities of language pose a major challenge, and identifying multiword aspects and their associated sentiment polarities hinders the development of effective models. To address the limitations of supervised approaches, unsupervised clustering can be employed to group similar aspect features. We propose an unsupervised hybrid architecture called the BERT + GPT‐2 fusion model. Our main technical contributions include aspect extraction utilizing the BERTopic model. Our approach is evaluated utilizing silhouette and coherence scores. To identify the optimal number of aspect clusters, cluster quality is measured by calculating the average silhouette and coherence scores. The clusters that maximize the average values are well‐defined clusters. The identified aspects are then used to train the BERT+GPT‐2 fusion model. The proposed technique was evaluated on the 515 K Hotel Reviews Data dataset, demonstrating its efficacy for sentiment analysis. Our fusion model outperformed state‐of‐the‐art approaches, including GPT‐2 and convolutional neural network baselines.