Integrating Taiwan financial BERT sentiment analysis with CNN-BiLSTM-SA model for stock prediction

In the stock market, investor sentiment can lead to rapid fluctuations, making it a crucial factor in stock prediction. This paper investigates the relationship between social sentiment in Taiwan and the fluctuations of the Taiwanese stock market. We employ BERT for domain-adaptive pre-training using Taiwanese stock market news to create a Chinese FinBERT model specialized in the Taiwanese financial sector. Using daily discussion content from the "PTT" stock forum as a reference for Taiwanese stock market sentiment, we fine-tune the Chinese FinBERT model to develop the Daily FinSentiment model, which analyzes social sentiment and calculates sentiment scores to serve as predictive features for stock market fluctuations. This paper proposes a CNN-BiLSTM-SA model that uses stock prices and sentiment scores as training data for stock market prediction. This model improves upon the CNN-BiLSTM-AM model by replacing the Attention Mechanism (AM) with Self-Attention (SA) for a more comprehensive understanding of the training data. The CNN layer extracts crucial features from the training data, the BiLSTM layer makes predictions based on these features, and the SA layer updates model weights by determining the impact of different temporal features on stock closing prices. Experimental results show that the CNN-BiLSTM-SA model achieves a 22% improvement in prediction accuracy over the CNN-BiLSTM-AM model, reaching an accuracy of 87.5% without incorporating sentiment features. When sentiment features are included, the accuracy further increases to 90.62%, demonstrating a significant enhancement in predictive performance. This paper integrates the CNN-BiLSTM-SA model with a LINE Bot, making the prediction results easily accessible and referenceable.