The effects of personalized feedback on participant behavior in an app-based diary study

In app-based data collection, researchers increasingly provide participants with information based on their data and study behavior through personalized feedback. Such personalized feedback is hypothesized to have positive effects on participant engagement and data quality. However, empirical evidence on these positive effects and research on the mechanisms on the effects that feedback has on participation remains scarce. We investigate the effects of providing personalized feedback on participation behavior in an app-based household budget diary study in the general population of the Netherlands. The timing of providing personalized feedback was varied through a random assignment to two experimental conditions: half of the participants received their personalized feedback instantly, and half of the participants received their personalized feedback after 14 days, at the end of the study. We used mobile app paradata to study how participants interact with their personalized feedback and whether providing personalized feedback leads to higher data quality and/or unwanted behavioral change. We find that most participants view theirpersonalized feedback and that visits to the screen presenting feedback are more likely to be longer when the feedback contains more new information for participants. We find no relationship between personalized feedback and data quality or unwanted behavioral change, but we do find a positive relation between data quality and the duration of viewing the feedback by participants. This finding suggests that the collection of in-app paradata can inform researchers about the whether the efforts of implementing personalized feedback which is costly, are worth undertaking. We conclude with suggestions for future research, including studying the forms of the feedback to be able to conclude whether personalized feedback should be used in app-based data collection.