The Many Reliabilities of Psychological Dynamics: An Overview of Statistical Approaches to Estimate the Internal Consistency Reliability of Intensive Longitudinal Data

Reliability is a key concept in psychology that has been broadly studied since the introduction of Cronbach's alpha, which is a measure of the internal consistency of a test. Despite its importance, this is a topic that is relatively understudied when dealing with intensive longitudinal data. In particular, when studying the psychological dynamics of affective states, there is no warranty that intensive longitudinal measurements are reliable. Given this, empirical researchers need tools to study and report the reliability of the scales used in intensive longitudinal research. In recent years, different approaches to estimate the reliability of the scales and the items used when studying psychological dynamics have been proposed. However, the advantages and disadvantages of each of these methods are unclear, making it difficult to determine when a certain approach would be preferred over the others. Specifically, these approaches estimate reliability indices based on different statistical models, such as linear multilevel analysis, vector autoregressive models, and dynamic factor models. Furthermore, while some methods involve estimating one reliability index for the scale that applies to the whole sample, others estimate person-specific reliability indices. This wide variety of approaches can provoke some confusion for empirical researchers. In this paper, we aim to highlight the advantages and disadvantages of different methods used to estimate the reliability of intensive longitudinal data. We also showcase their application with empirical data.