Comparing MCMC algorithms for the horseshoe prior

The horseshoe prior is notably one of the most popular priors in sparse regression models, where only a small fraction of coefficients are nonzero.The parameter space of the horseshoe prior is much smaller than that of the spike and slab prior, so it enables us to efficiently explore the parameter space even in high-dimensions.However, on the other hand, the horseshoe prior has a high computational cost for each iteration in the Gibbs sampler.To overcome this issue, various MCMC algorithms for the horseshoe prior have been proposed to reduce the computational burden.Especially, Johndrow et al. (2020) recently proposes an approximate algorithm that can significantly improve the mixing and speed of the MCMC algorithm.In this paper, we compare (1) the traditional MCMC algorithm, (2) the approximate MCMC algorithm proposed by Johndrow et al. ( 2020) and (3) its variant in terms of computing times, estimation and variable selection performance.For the variable selection, we adopt the sequential clustering-based method suggested by Li and Pati (2017).Practical performances of the MCMC methods are demonstrated via numerical studies.