Various Optimized Artificial Neural Network Simulations of Advection-Diffusion Processes

Abstract The aim of this research is to describe an artificial neural network (ANN) based method to approximate the solutions of the natural advection-diffusion equations. Although the solutions of these equations can be obtained by various effective numerical methods, feed forward neural network (FFNN) techniques combined with different optimization techniques offer a more practicable and flexible alternative than the traditional approaches to solve those equations. However, the ability of FFNN techniques to solve partial differential equations is a questionable issue and has not yet been fully concluded in the existing literature. The reliability and accuracy of computational results can be advanced by the choice of optimization techniques. Therefore, this study aims to take an effective step towards presenting the ability to solve the advection-diffusion equations by leveraging the inherent benefits of ANN methods while avoiding some of the limitations of traditional approaches. In this technique, the solution process requires minimizing the error generated by using a differential equation whose solution is considered as a trial solution. More specifically, this study uses a FFNN and backpropagation technique, one of the variants of the ANN method, to minimize the error and the adjustment of parameters. In the solution process, the loss function (error) needs to be minimized; this is accomplished by fitting the trial function into the differential equation using appropriate optimization techniques and obtaining the network output. Therefore, in this study, the commonly used techniques in the literature, namely gradient descent (GD), particle swarm optimization (PSO) and artificial bee colony (ABC), are selected to compare the effectiveness of gradient and gradient-free optimization techniques in solving the advection-diffusion equation. The calculations with all three optimization techniques for linear and nonlinear advection-diffusion equations have been run several times to obtain the optimum accuracy of the results. The computed results are seen to be very promising and in good agreement with the effective numerical methods and the physics-informed neural network (PINN) method in the literature. It is also concluded that the PSO-based algorithm outperforms other methods in terms of accuracy.