Feasibility of Digital Memory Assessments in an Unsupervised and Remote Study Setting

Sensitive and frequent digital remote memory assessments via mobile devices hold the promise to facilitate the detection of cognitive impairment and decline. However, in order to be successful at scale, cognitive tests need to be applicable in unsupervised settings and confounding factors need to be understood. This study explored the feasibility of completely unsupervised digital cognitive assessments using three novel memory tasks in a Citizen Science project across Germany. To that end, the study aimed to identify factors associated with stronger participant retention, to examine test-retest reliability and the extent of practice effects, as well as to investigate the influence of uncontrolled settings such as time of day, delay between sessions or screen size on memory performance. A total of 1,407 adults (aged 18-89) participated in the study for up to 12 weeks, completing weekly memory tasks in addition to short questionnaires regarding sleep duration, subjective cognitive complaints as well as cold symptoms. Participation across memory tasks was pseudorandomized such that individuals were assigned to one of three memory paradigms resulting in three otherwise identical sub-studies. One hundred thirty-eight participants contributed to two of the three paradigms. Critically, for each memory task 12 independent parallel test sets were used to minimize effects of repeated testing. First, we observed a mean participant retention time of 44 days, or 4 active test sessions, and 77.5% compliance to the study protocol in an unsupervised setting with no contact between participants and study personnel, payment or feedback. We identified subject-level factors that contributed to higher retention times. Second, we found minor practice effects associated with repeated cognitive testing, and reveal evidence for acceptable-to-good retest reliability of mobile testing. Third, we show that memory performance assessed through repeated digital assessments was strongly associated with age in all paradigms, and individuals with subjectively reported cognitive decline presented lower mnemonic discrimination accuracy compared to non-complaining participants. Finally, we identified design-related factors that need to be incorporated in future studies such as the time delay between test sessions. Our results demonstrate the feasibility of fully unsupervised digital remote memory assessments and identify critical factors to account for in future studies.