Bayesian Implementation of the Factor‐Analytic Mixed Model and Application to Embeddings

Mixed models and neural networks each offer complementary frameworks for prediction. Theoretically grounded in inference, mixed models can unveil latent variance structure notably with the factor‐analytic approach. In this article, we propose to bridge the gap between the two frameworks, leveraging embedding data from a neural network encoder to structure the covariance matrix in a mixed model. First, we implemented a Bayesian factor‐analytic mixed model and challenged it using simulations. The model showed sensitivity to the prior, with a propensity to overfitting under some hyperparameterizations. However, with an appropriate hyperparameterization it showed high predictive performance and a good ability to infer the latent structure (accuracy between 0.881 and 0.919 on the two first factorial axes). Using a challenging design in which each individual was observed a single time in a single environment, we showed that the embedding information enabled prediction of unobserved individuals. A small but significant ( p ‐value < 0.05) amount of real bibliometric data variance was captured using this factor‐analytic component applied within a negative‐binomial mixed model. These results call for further investigation of the use of the mixed model framework to analyse embedding data.