Searching in ultra high dimensional sparse model spaces: New performance tests and boosting Gibbs sampling algorithms

Abstract In the context of variable selection in sparse settings, we present a novel class of experiments. These are based on the notion of contaminating real data sets with artificial spurious covariates in such a way that exact solutions can be easily computed. Such exact responses provide a direct and compelling way to evaluate the performance of search methods on model spaces of arbitrary cardinality. We apply this tool to Gaussian regression models, an important statistical problem that has benefited from the emergence of many new search methods in recent years. We also contribute to this catalog by revisiting classical Gibbs sampling algorithms proposing new implementations that take advantage of sparsity. Despite their simplicity, the resulting methods are very competitive and fully automatic. We use a real genetic dataset to illustrate and motivate the various procedures presented in this research.