Benchmark evaluation reveals pervasive data leakage in standard phishing URL datasets, highlighting the necessity of strict feature auditing for realistic machine learning security models.
Phishing remains one of the highest-volume initial access vectors in modern cyberattacks, and supervised machine learning is widely used to classify malicious URLs. This paper presents a controlled comparison of Logistic Regression and Random Forest on the PhiUSIIL phishing URL dataset, using both a 17-feature lexical URL-only configuration and a full 50-feature configuration incorporating rendered-page attributes. Although Random Forest achieved perfect performance on the full feature set, further analysis revealed significant data leakage and collection artifacts within the benchmark. In particular, URLSimilarityIndex acts as a near-deterministic proxy for the class label, while rendered-page features reflect differences in the collection of live and unreachable phishing pages. A leakage-controlled URL-only configuration produced a more defensible accuracy of 99.65% with Random Forest and 98.57% with Logistic Regression. The study demonstrates the importance of leakage auditing and realistic evaluation when developing machine-learning-based phishing detection systems and provides reproducible code and configurations for further research.
No takes yet. Share an insight, caveat, or question.
Sai Nithin Erigineni (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: