PANH: A Padé Approximated Non-Linear Hyperbolic Activation Function for Physics-Informed Neural Networks

Physics-informed neural networks frequently suffer from optimization difficulties and vanishing gradients when relying on standard saturating activation functions. To address these limitations, we introduce an adaptive rational activation function grounded in theoretical approximation principles that guarantees non-vanishing gradients while maintaining extreme smoothness. Empirical evaluations across diverse differential equation benchmarks demonstrate that this approach autonomously adapts to various functional regimes, significantly improving convergence and accuracy, particularly for highfrequency and multi-scale problems.