What question did this study set out to answer?

This research aims to enhance Bayesian neural networks through a novel approach integrating differential evolution and MCMC.

February 23, 2026Open Access

DE-BNN: An evolutionary approach to Bayesian neural network posterior sampling

Key Points

This research aims to enhance Bayesian neural networks through a novel approach integrating differential evolution and MCMC.
Developed DE-BNN for training Bayesian neural networks.
Integrated differential evolution with MCMC to create a DE-MCMC framework.
Applied DE-BNN to industrial problems like power load forecasting and concrete strength prediction.
Compared DE-BNN's performance with variational inference and Hamiltonian Monte Carlo methods.
DE-BNN demonstrated superior accuracy and performance in constructing Bayesian neural networks.
Provided better probabilistic predictions with credible intervals covering actual values.
Achieved the lowest ECPE values in testing on a larger superconductor dataset.

Abstract

A Bayesian neural network (BNN) extends traditional neural networks (NNs) with posterior inference by viewing weights and biases of the NN as random variables, and has significant advantages over standard NNs. The BNN can be further enhanced by using Markov Chain Monte Carlo (MCMC), but it is still very challenging due to issues like poor scalability, non-convergence, and others. In this work, we propose a novel nonparametric approach called DE-BNN to train BNN by integrating differential evolution (DE) and MCMC as DE-MCMC. DE is a robust stochastic optimization technique with good scalability that has been either directly applied to diverse areas, or used as an alternate training mechanism in building NN instead of backpropagation, forming a neuroevolution algorithm. The sequence of solution candidates produced by DE can be treated as a modified Markov Chain and thus the proposal distribution generator in the DE-based MCMC method. We apply this DE-MCMC to train our DE-BNN and our results demonstrate its efficacy in constructing BNN in neuroevolution regression problems We then applied it to two industrial problems for test: hourly power load forecasting in the energy industry and concrete compressive strength prediction in civil engineering. Further, we extend our DE-BNN into a probabilistic BNN by providing two point estimates: a mean value prediction along with a credible interval and a prediction using the mode of each posterior distribution. We also directly compare the results of DE-BNN with a variational inference (VI) BNN and Hamiltonian Monte Carlo (HMC) BNN and our results show the DE-BNN achieve close or better results in accuracy and performance but a much better probabilistic prediction that covers the actual values. A final test is run with a larger model applied to the superconductor dataset. DE-BNN again emerges on top with the lowest ECPE values and competitive test fitness.

Read Full Paperexternally

Mark Helpful

Bookmark

Relay

View Full Paper