BieLU: A Bounded Exponential-Kernel Activation for Stable Probabilistic Forecasting in Deep and Residual Networks

We introduce BieLU, a bounded activation with anexponential kernel, f (x) = x · exp(−κ|x|), with a learnabledecay rate κ. The function is odd, smooth with unit slope at theorigin, and bounded with supremum 1/(κe), and it admits closed-form second moments. Boundedness has a direct consequencefor deep architectures. A residual network is a forward-Eulerdiscretisation of a Neural ordinary differential equation, whosesolution requires a bounded Lipschitz vector field. An unboundedactivation can drive the discretised trajectory to diverge. Weconfirm this on a probabilistic forecasting task over intradayfinancial returns across three architectures of increasing depthand three instruments. On a shallow multilayer perceptron alltested activations agree. On convolutional and residual heads theunbounded ReLU and GELU destabilise: the held-out negativelog-likelihood of ReLU exceeds its stable value by several ordersof magnitude on the residual head, while bounded BieLU staysstable and calibrated. The instability affects both the predictivescale and the predicted median, and it worsens with depth andwith poorer input, as the analysis predicts. A bounded activationoutside the BieLU family, x · sech(x), shows the same stability,which points to boundedness as the operative property. A featureablation shows that a common scale-volatility correlation islargely feature-driven.