Application of Artificial Intelligence and Machine Learning in Data Transmission over Telecommunication Networks

In modern and future telecommunication systems-particularly within 5G-Advanced and emerging 6G networks-data transmission optimization remains a critical challenge in the context of the terahertz (THz) spectrum, ultra-reliable lowlatency communications (URLLC), and massive Internet of Things (mMTC).Analytical models relying on traditional Shannon-Kotelnikov theory fall short in achieving fully efficient resource management and signal recovery under dynamic environments characterized by high fading, multipath propagation, and intense interference.This research provides a comprehensive analysis of the integration of artificial intelligence (AI) and machine learning (ML) algorithms across all stages of data transmission.It proposes a hybrid physicalstatistical framework that merges the mathematical models of the physical layer with data-driven ML methods based on Edge AI.As part of this study, a 5G HetNet topology comprising macrocells, pico-cells, and IoT nodes was simulated using NS-3 (Network Simulator 3).The performance of SVM, Decision Trees, K-means, and PCA was evaluated alongside reinforcement learning models (Q-learning, Deep Q-Networks -DQN).The obtained simulation results demonstrate that the hybrid ML approach, and specifically deep reinforcement learning (RL), enables reducing network latency by up to 22%, increasing overall throughput by 18%, raising the accuracy of anomaly and security attack detection to 94%, and significantly enhancing energy efficiency compared to traditional routing and resource allocation methods.