Cosine similarity based hybrid attention mechanism for log anomaly detection

System logs contain rich information for monitoring system operation. Although existing deep learning models have achieved significant success in log anomaly detection, sequence-type log data have not yet clearly learned the local and global relationships within sequences, creating a bottleneck in the development of log anomaly detection. To address this issue, we propose a cosine similarity-based hybrid attention mechanism log anomaly detection algorithm, named cosHALog. Firstly, after log parsing, we select three keywords from every log template based on inverse document frequency (IDF) algorithm, use Bidirectional Encoder Representation from Transformers (BERT) to extract the semantic information in templates, and then apply Principal Component Analysis (PCA) for dimensionality reduction. Subsequently, three parallel one-dimensional convolutional neural networks (CNNs) are employed to learn local relationships in the sequence, combined with a simple additive attention mechanism to assist CNNs in learning. Finally, inspired by cosine similarity and Knowledge Distillation, we optimize the Vaswani attention mechanism to make it a parameter-free attention mechanism and use cosine similarity to measure the relationships between sequence nodes, thereby facilitating Gate Recurrent Unit (GRU) to capture global relationships. To evaluate the performance of cosHALog model, we compare it with the other eight hybrid attention mechanisms to select the most suitable attention mechanisms for learning local and global relationships. Meanwhile, we also compare cosHALog with six benchmark models. The experimental results show that, compared with DeepLog, CNN, LogAnomaly, LogRobust, LogBERT and LogGPT, our proposed method achieves an average improvement of 4.20%, 1.21%, 5.30%, 2.79%, 16.67% and 8.89% in F1-score on Hadoop Distributed File System (HDFS) dataset, respectively. All experiments in this article are conducted on HDFS, BGL and Thunderbird datasets. Our source code and datasets are freely available on https://github.com/Lxc2git/c0sHALog.git .