Bayesian model averaging of (a)symmetric item response models in small samples

This is the authors' abstract. We don't add key points for this paper.

Asymmetric IRT models present theoretically desirable features, but often require large sample sizes for stable estimation due to additional item parameters. When applying item response theory (IRT) to small samples, it is often the case that only models with relatively few item parameters can be reliably estimated. Two recently developed asymmetric IRT models, the negative log-log and the complementary log-log, allow for different IRF shapes compared to conventional IRT models and can be fit with small samples. In this paper, we propose Bayesian model averaging (BMA) of simple symmetric and asymmetric IRT models to explore item asymmetry and to flexibly estimate IRFs in small samples. We also consider model averaging at both the item level and the test level. We first show the feasibility of the approach with an empirical example. Then, in a simulation study involving complex data-generating conditions and small sample sizes (i.e., 100 and 250), we show that averaging methods recover asymmetry in the data-generating process and consistently outperform model selection and kernel smoothing. The methods proposed in this study are a practical alternative to more complex asymmetric IRT models and may also be a useful method in exploratory semi-parametric IRT analysis.