Self-evolving meta-learning neural ordinary differential equations: A novel artificial intelligence-driven approach for solving differential equations

Solving ordinary differential equations (ODEs) constitutes a fundamental problem for many scientific and engineering disciplines, particularly for stiff, high-dimensional problems, or problems with changing dynamics. Consequently, traditional numerical solvers, such as the Runge–Kutta methods, suffer from the drawback of computational inefficiency, numerical in stability and a lack of good generality across a range of problems. However, some of these limitations had been overcome with the advent of deep learning methods, leading to the development of neural ordinary differential equations (Neural ODEs) methods. However, the Neural ODEs fixed architecture makes it less versatile and especially in cases of solving complex and nonlinear ODEs problems. To address these problems, this research proposes self-evolving meta-learning neural ODE (SEML-NODE), a new approach of meta-learning and evolutionary computing for making a self-adjusting neural solver that is capable of evolving its own architecture during training. The main novelty of SEML-NODE is the self-evolution mechanism that enables the neural network's structure to expand and/or contract according to error feedback so that both the accuracy and generalization are improved. The adaptive training process renders the new extrapolation method to have superior performance on the stiff or chaotic ODEs. In this work, we evaluate SEML-NODE on benchmark problems, i.e. logistic growth model, Kepler's problem, and Fisher's equation, which contain difficulty such as nonlinearity, stiffness, and spatial-temporal dynamical problem. The results show that SEML-NODE is consistently better than state-of-the-art methods, such as fixed Neural ODEs and physics-informed neural networks, achieving significantly improved results in terms of errors on all benchmark tasks. Since the framework is able to dynamically change architecture depending on complexity of problem, by solving more complex, time-varying systems is very efficient, which is a problem for traditional solvers. While the current dynamic network expansion has a big initial cost in computed power, it becomes more efficient as the network training progresses by eliminating unnecessary network complexity and only expanding as needed. However, the improved optimization of self-evolution parameters (such as the error threshold of network expansion) has been challenging to prevent overfitting. To overcome this, for future work, the optimization of these parameters and the extension of the SEML-NODE to partial differential equations will be the focus. Besides, reinforcement learning will also be explored to further optimize the architecture. The proposed SEML-NODE framework greatly improves the adaptive ODE solver development and may find many applications in real-world areas such as climate prediction, drug discovery, and financial prediction, where the traditional numerical methods cannot provide efficient solutions.