MiniLM-CNN-LSTM: A Lightweight Hybrid Transformer Model for Malicious URL Detection

Phishing and malicious websites are a serious threat on the internet. Attackers use fake links to trick users and steal their private information. Detecting these links is difficult because attackers change their tricks often. Many old methods cannot detect new or hidden threats. Some recent models use deep learning (DL), but they are large, slow, and hard to use in real-time systems. In this paper, we present a lightweight and accurate model called MiniLM-CNNLSTM. It combines a small transformer model (MiniLM) with a hybrid DL network using Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) layers. The transformer learns the meaning of URLs. The CNN finds important patterns. The LSTM captures the order of characters. We also add handcrafted features that help the model detect tricky URLs. We test our method on two public datasets: the Phishing Site URLs dataset and the Malicious URLs dataset from Kaggle. We use 3-fold cross-validation and early stopping to ensure fair and stable results. The MiniLM-CNN-LSTM model outperformed previous benchmarks by achieving an average three-fold cross-validation accuracy of 98.98%, a precision of 98.63%, a recall of 98.29%, an F1-score of 98.46%, and a false positive rate of 0.68%. The proposed model has a higher accuracy, precision, recall, F1-score and a lower false positive rate, which enhances the accuracy by 1.88, precision by 3.77, recall by 4.17 and decreases the false positive rate by 61.58% compared with the strongest baseline (Distil BERT + CNN-LSTM), showing significant practical improvements. The results show that our approach is fast, small, and highly effective. It can detect phishing and malicious links with high accuracy. This makes it a good choice for real-time security systems like browsers, email filters, or firewalls.