An investigation of the temporal dynamics of careless responding across different populations in experience sampling data

The experience sampling method (ESM) has become a popular tool across various fields ofpsychology. However, the intensive nature of ESM raises concerns about carelessresponding, where participants respond to questionnaires without paying sufficientattention to them. To better understand careless responding in the context of ESM, thisstudy investigated its temporal dynamics across three commonly used sample types(community, student, clinical). We leveraged four careless responding indicators fromprevious research — response time, within-beep standard deviation, an inconsistencyindex, and occasion-person correlation — and used univariate and multivariate multilevelmodels to examine their trajectories over time, both over the course of the study and overthe course of a day. Our results demonstrate that careless responding is not a stablephenomenon but changes over time, with evidence for increases in carelessness across daysand non-stationarity within days across the different samples. Furthermore, our findingsimply that the indicators differ in their ability to capture carelessness: while response time,within-beep standard deviation, and the inconsistency index show potential for capturingcarelessness, occasion-person correlation appears unsuited for an ESM context. Thepresence of few and small associations among the indicators indicates that they flagdistinct and independent kinds of carelessness and thus complement one another, making itadvisable to use multiple indicators where possible. Overall, our findings underscore theimportance of considering the temporal dynamics of carelessness in ESM studies anddemonstrate the usefulness of the indicators in assessing it.