What question did this study set out to answer?

The study aims to evaluate activation functions for initial value problems and utilize Automatic Differentiation for precise gradient calculations.

May 26, 20260 citationsOpen Access

Artificial Neural Networks with Different Activation Functions to Solve Higher-Order Ordinary Differential Equations

PJPatil JayasreeVijayanagara Sri Krishnadevaraya University ARA Padmanabha ReddyVijayanagara Sri Krishnadevaraya University KNK NagaveniBall (France)

Key Points

The study aims to evaluate activation functions for initial value problems and utilize Automatic Differentiation for precise gradient calculations.
Systematic assessment of various activation functions for solving initial value problems (IVPs).
Implementation of Automatic Differentiation in the gradient evaluation process to enhance numerical stability.
Validation through benchmark higher-order IVP problems compared to existing numerical methods.
The proposed ANN-AD method achieves maximum absolute errors below 10−6 for benchmark cases, indicating superior accuracy.
Stable convergence behavior observed across first, second, and fifth-order IVPs without specific tuning requirements.
Demonstrates flexibility and efficiency for solving higher-order differential equations, with potential for extension to complex boundary value problems.

Abstract

Abstract Objectives: To make (i) a systematic assessment of activation functions for solving initial value problems (IVPs) and ii) the use of Automatic Differentiation (AD) for accurate gradient evaluation in Gradient Descent (GD). Method: Unlike conventional backpropagation approaches that rely on manually derived gradients, the proposed AD-based framework enables precise derivative evaluation using the chain rule, improving numerical stability and convergence efficiency. The performance of the proposed method is validated through several benchmark higher-order IVP test problems and compared with existing numerical techniques including Regression-Based Weight initialization (RBW), Chebyshev Wavelet Collocation Method (CWCM), and Haar Wavelet Collocation Method (HWCM). Findings: In the present study, an enhanced ANN-based methodology incorporating Automatic Differentiation (AD) is developed to solve higher-order initial value problems (IVPs) with improved accuracy and convergence. The numerical experiments indicate that the proposed ANN-AD approach achieves maximum absolute errors, depending on the order and complexity of the test problems. In several benchmark cases, the method consistently attains errors below 10−6, demonstrating superior approximation accuracy compared to existing ANN-based approaches reported in the literature. Furthermore, the proposed method exhibits stable convergence behaviour across first, second and fifth-order IVPs without requiring problem-specific tuning. These results confirm that the proposed methodology provides an accurate, flexible and computationally efficient framework for solving higher-order differential equations. Additionally, its robustness and generalization capability suggest strong potential for extension to more complex boundary value problems and nonlinear differential systems in future work. Novelty: This work combines Automatic Differentiation– driven gradient optimization with activation-function sensitivity analysis in a unified ANN framework for higher-order IVPs, a combination not systematically explored previously. Keywords: Differential equations; Neural network; Activation function; Automatic differentiation; Gradient descent

Perguntar à IA

Bookmark

View Full Paper