Physics-Guided Bayesian Neural Networks and Their Application in ODE Problems

Abstract Deterministic Neural Networks (NNs) have been widely adopted to represent system behaviors in various engineering problems, which have been criticized for providing overconfident predictions. This challenge, i.e., overfitting, refers to a model that captures data patterns and random noise during the learning process, leading to an inferior predictive capability in the unobserved domains. As known to all, Bayesian inference offers an effective way for uncertainty quantification (UQ) to help enhance model predictions and offer informed decision-making. Integrating deterministic NN models with Bayesian inference provides a way to fast simulate and quantify both data and model uncertainty simultaneously. Thereby, Bayesian Neural Networks (BNNs), a combination of standard NNs with Bayesian philosophy, introduce probability to the network parameters, i.e., biases and weights. In BNNs, predictive uncertainty reflects how confident model predictions are regarding the corresponding credible intervals, which helps identify overfitting and interpreting abnormal predictions. Another advantage is that BNNs naturally implement regularization by integrating over the weights such that BNNs can reach good generalization when data availability is limited. BNNs rely heavily on observations as a purely data-driven approach. Besides, standard BNNs have relatively complex network structures, which makes it hard to provide detailed interpretability on model predictions. Researchers share concerns about their predictions that may violate real-world phenomena. The combination of physics knowledge with the optimization objective can improve the adherence of predictions with the known physical principles. Physics laws are usually embedded as constraint functions in the process of optimization. With the help of constrained optimization, model predictions keep consistent with the known physical principles. The authors refine BNNs with physics-guided information, i.e., Physics-Guided BNNs (PG-BNNs), to reduce dependence on data, improve model interpretability, and offer reliable model predictions. PG-BNNs interpret the known physics knowledge as constraint functions to better guide optimization processes, especially for case studies with sparse and limited data sets. A case study of an ordinary differential equation (ODE) problem is carried out to compare the performance of PG-BNN and unconstrained BNN models and demonstrate the effectiveness of PG-BNN.