Optimized machine learning approach for detecting TCP exhaustion attacks in modbus-TCP/IP networks

Abstract The Modbus TCP/IP protocol, widely adopted in industrial communications, lacks essential security features, making it vulnerable to cyberattacks such as TCP Connection Exhaustion. This paper proposes a machine learning-based detection framework using the Random Forest (RF) algorithm to identify malicious traffic in Operational Technology (OT) networks. A simulated testbed was created using virtual machines to emulate Modbus server-client communication under normal and attack conditions. Our model achieved F1-score of 99.83 %, precision of 99.9 %, and recall of 99.7 %, clearly demonstrating its accuracy and robustness. These results validate the proposed approach as a lightweight, real-time, and effective intrusion detection system suitable for resource-constrained industrial environments.