Meta-Learning-Based Adaptive Hyperparameter Tuning for Nonlinear Compensation in Optical Transmission

To break the nonlinear Shannon capacity bottleneck, next-generation coherent systems are widely adopting probabilistic constellation shaping (PCS) and ultra-wideband multiplexing. However, these critically exacerbate nonlinear impairments. While neural network-based nonlinear equalizers (NNLEs) offer a promising mitigation path, their practical deployment is fundamentally crippled by the prohibitive manual overhead and fragile convergence of hyperparameter tuning (HT). We introduce a meta-learning-based adaptive HT framework (MAHT) that breaks this barrier by establishing a differentiable co-optimization across training dynamics and data rebalancing, enabling NNLEs to self-adapt without human intervention. We experimentally demonstrate an S+C+L-band DP-PCS-128QAM transmission over 150 km fiber, attaining an achievable information rate (AIR) of 215.35 Tbps. Moreover, MAHT delivers exceptional weight efficiency, outperforming conventional methods by nearly 150%, and automatically optimizes across multiple channels. This work establishes a self-optimizing, resource-efficient paradigm that bridges machine learning with practical high-speed optical networks, paving the way for truly autonomous transceiver systems.