PhishSieve: Scalable and Adaptive Phishing Detection on a Live Stream

We propose a new paradigm for thwarting phishing attacks through zero-day removal of phishing URLs from the continuous influx of new URLs on the Internet. This approach demands a high-throughput, high-recall, and high-precision system, capable of filtering millions of URLs daily. Processing a dynamic online stream also enables real-time observation of emerging phishing tactics and requires new adaptive methods to keep the machine learning model effective against the evolving threats. In this paper, we design a phishing detection system that supports the new paradigm and addresses the above challenges and opportunities. While most current anti-phishing ML & DL methods are trained on static datasets, attackers rapidly update their phishing tactics to evade these models, rendering models that are even a few months old largely ineffective. Instead, we build upon a live stream of newly created websites, such as CertStream. To efficiently process the millions of URLs appearing daily, we adopt a two-stage architecture that balances speed and accuracy. Our primary focus lies in the first stage, a lightweight model that rapidly processes the high volume of URLs, filtering out clearly benign links and isolating potentially suspicious ones for further inspection. This stage leverages unique, resilient, network-based indicators (e.g., IP TTL, DNS and HTTP response times) which ensure additional robustness against adversarial manipulations. In the second stage, resource-intensive deep learning methods are applied to validate or dismiss the flagged URLs with high confidence. This two-stage design enables the system to identify phishing URLs with high precision. Moreover, its efficiency boosts the effective recall rate by allowing a significantly larger portion of the live URL stream to be processed and analyzed. We have conducted a field study over a course of several months to showcase the robustness of the trained model. We also evaluated a self-updating framework addressing model drift seamlessly in realistic scenarios.