Penalized mixed models to adjust for batch effects and unobserved confounding in high dimensional regression

Confounding can lead to spurious associations. Typically, one must observe confounders in order to adjust for them, but in high-dimensional settings, recent research has shown that it becomes possible to adjust even for unobserved confounders. The methods for carrying out these adjustments, however, have not been thoroughly investigated. In this study, we derive a confounding framework in which the signal, bias, and variability can be cleanly partitioned and quantified, thereby enabling simulations in which one varies the bias-to-signal ratio while holding the signal-to-noise ratio fixed. Using this construction, we demonstrate the impact of the amount and complexity of unobserved confounding on the performance of competing methods, including the LASSO, principal components LASSO (PC-LASSO), and penalized linear mixed models (PLMMs). We identify scenarios in which each method outperforms the others and find that overall, PLMM is the most robust approach.