Monte Carlo approximation of the logarithm of the determinant of large matrices with applications for linear mixed models in quantitative genetics

Abstract Background Likelihood-based inferences such as variance components estimation and hypothesis testing need logarithms of the determinant (log-determinant) of high dimensional matrices. Calculating the log-determinant is memory and time-consuming, making it impossible to perform likelihood-based inferences for large datasets. Results We presented a method for approximating the log-determinant of positive semi-definite matrices based on repeated matrix–vector products and complex calculus. We tested the approximation of the log-determinant in beef and dairy cattle, chicken, and pig datasets including single and multiple-trait models. Average absolute relative differences between the approximated and exact log-determinant were around 10 –3 . The approximation was between 2 and 500 times faster than the exact calculation for medium and large matrices. We compared the restricted likelihood with (approximated) and without (exact) the approximation of the log-determinant for different values of heritability for a single-trait model. We also compared estimated variance components using exact expectation–maximization (EM) and average information (AI) REML algorithms, against two derivative-free approaches using the restricted likelihood calculated with the log-determinant approximation. The approximated and exact restricted likelihood showed maxima at the same heritability value. Derivative-free estimation of variance components with the approximated log-determinant converged to the same values as EM and AI-REML. The proposed approach is feasible to apply to any data size. Conclusions The method presented in this study allows to approximate the log-determinant of positive semi-definite matrices and, therefore, the likelihood for datasets of any size. This opens the possibility of performing likelihood-based inferences for large datasets in animal and plant breeding.