Discriminatory Accuracy of a Random effect in Multilevel Logistic Regression.

INTRODUCTION: Multilevel logistic regression analyses are applied in a variety of different scenarios, not least for the study of geographic differences in health. Oftentimes these studies are intended as a basis for public health interventions and/or individual lifestyle recommendations. However, results from these multilevel analyses are not easily translated into measures that inform on the potential effectiveness of future interventions, or on the accuracy of individual health predictions, such as sensitivity, specificity and measures derived therefrom. The project aim is to develop a method for translating multilevel analyses results into measures of discriminatory accuracy, such as the area under the receiver operating characteristics curve (AUC-ROC). The method will help elucidate the usefulness of random second level factors, such as geographical areas, schools, hospitals etc, as targets of public interventions and/or as predictors of individual health outcomes. METHODS: We examine the distributions of the random effects among cases and non-cases in different scenarios by means of theoretical investigation and simulation. The distributions are subsequently used to produce ROC curves and corresponding AUC-ROC measures. We also illustrate a mathematical relation between the AUC-ROC and ICC of the random effect under certain conditions. Results are applied to empirical data on hospital differences in mortality following breast cancer diagnoses (BCD) in Sweden 2005–2012.