Evaluation of AES-256 encryption and machine learning for securing GSM communications against sniffing attacks

GSM (Global System for Mobile Communications) networks are becoming exposed to security attacks more frequently because of their inefficient encryption algorithms, like A5/1 and A5/2. These protocols are vulnerable to man-in-the-middle attacks, including brute-force and side-channel attacks that intercept sensitive user data such as voice calls, text messages, and location data. The purpose of the study is to advance GSM security by adding to it strong encryption methods and sophisticated detection schemes. In particular, the study examines the viability of AES-256 and machine learning-based algorithms in real-time detection of sniffing attacks. The proposed approach will examine the feasibility of attacks in the real-time sense, modeling GSM environments using Software Defined Radio (SDR) tools (RTL-SDR and BladeRF) to generate encrypted traffic. AES-256 is added to overcome old ciphers, and Support Vector Machine (SVM) models are trained to recognize unusual traffic patterns, which are an indicator of sniffing attacks. The findings indicate that by using the AES-256 in conjunction with an SVM detection system, we were able to provide a high level of decryption protection (5%) compared to 95.2 with minimal latency (5 s). The main contribution of the current research is the development of a fully hybrid GSM security system incorporating a combination of high-level encryption and adaptive detection to provide high protection, even under resource-constrained conditions (such as IoT). To ensure more security of data confidentiality, integrity, and availability from all forms of encroachment by cyber threats in future GSM infrastructure, it is recommended that such compounded methods be embraced.