Behavioral Insights of SARSA-Based Reinforcement Learning for IPv6 Intrusion Classification

Intrusion Detection Systems (IDS) are crucial for network security, but traditional models like signature-based and supervised learning approaches require frequent updates or retraining. This study introduces a reinforcement learning-based IDS using SARSA for IPv6 intrusion detection. The SARSA agent with epsilon 0.1 (E-0.1) performed best: optimal detection at step 11, 193,453 correct and 25,014 incorrect actions, total reward 842,195, average reward 3.855 per epoch, outperforming E-0.5 and E-0.9. Furthermore, a comparison of Q-Table exploitation between Q-Learning and SARSA indicates that both algorithms yield similar results. However, SARSA is preferable in security contexts because its more realistic approach results in a lower false alarm rate. This behavioral distinction is demonstrated by a -1.2109 value difference and a substantially lower mean state value for SARSA (4.903) compared to the more optimistic Q-Learning (16.310). To ensure robustness, the results were validated using different random seeds, and E-0.1 consistently produced stable total rewards with minimal variance. Additionally, Shannon entropy confirmed the agent's stable decision-making behavior, where E-0.1 showed faster convergence, indicating efficient learning. Altogether, these findings suggest that the SARSA-based IDS offers a more adaptive and reliable solution for intrusion detection, particularly in dynamic IPv6 environments.