GWKBR: a novel method integrating machine learning and Bayesian inference framework to improve genomic prediction accuracy

Abstract Non-additive genetic effects pose significant challenges to traditional genomic prediction methods. Inspired by the ability of kernel-based machine learning methods to capture non-additive effects and the accurate genomic prediction of Bayesian methods, we developed a novel genomic prediction method, Genome-Wide Association Studies-Weighted Gaussian Kernel Bayesian Regression (GWKBR), which introduces a new form of covariance structure prior distribution and integrates machine learning techniques (weighted Gaussian kernel regression and Bayesian optimization), Bayesian inference, Restricted Maximum Likelihood (REML), genome-wide association studies (GWAS), and cross-validation process. By constructing a weighted Gaussian kernel, the method effectively captures non-additive effects and account for the relative importance of different single nucleotide polymorphisms (SNPs). We assessed its genomic prediction accuracy against six existing genomic prediction methods (genomic best linear unbiased prediction (GBLUP), BayesB, BayesR, Stochastic-Lanczos-Expedited Mixed Models (SLEMM), polynomial kernel ridge regression (KPRR), and deep neural network genomic prediction (DNNGP)) using simulated data, human data, and multiple plant and animal datasets (spruce, wheat, maize, cattle, and pigs). Our results indicated that GWKBR could flexibly adapt to diverse genetic architectures and deliver robust genomic predictions, with its advantage being particularly evident in plant datasets strongly influenced by non-additive effects. Among all 23 traits analyzed, GWKBR achieved the highest average accuracy for 13 traits and the second-highest for seven traits. Our findings demonstrated the reliability and robustness of GWKBR for genomic prediction in human and diverse plant and animal species. The GWKBR software is available at https://github.com/Wangxuer521/GWKBR/.