Ongoing Refinement and Adaptation are Required to Address Participant Deception in Online Nicotine and Tobacco Research Studies

Online research offers many benefits to nicotine and tobacco research, including the ability to conveniently and cheaply recruit large numbers of participants over a wide geographic area. However, online research also introduces an increased risk of participant deception and fraud.1–3 Heffner et al. introduced valuable techniques for detecting fraud that can be applied to a wide variety of study protocols.3 We offer an example of detecting and responding to participant deception and adapting to changing patterns of fraud in an online cohort study of adolescents and young adults. The goal of the Policy and Communication Evaluation (PACE) Vermont Study is to understand the impact of state-level policies and communication campaigns on substance use beliefs and behaviors in young Vermonters.4 Study measures and findings are aimed at informing state-level prevention programming.5,6 Using a cohort design, participants aged 12–25 years are recruited via multiple methods (including online advertisements) to visit a study website and complete online eligibility screening and informed consent.4 Between 2020 and 2021, three periods of enrollment were open during which participants were screened through Qualtrics surveys, and then manually screened by a member of our study team. Automated survey features were employed within Qualtrics to prevent multiple screener attempts, identify potential duplicates, and detect bots7; screeners also included CAPTCHA verification and one of three simple math problems which served as an attention check and were randomized to ensure that the same response was not copied across surveys. In consultation with our IRB, we included language in our online informed consent about discontinuing participants for providing information in the surveys that we believed to be intentionally falsified. The systematic method of screening participants for deception developed in our pilot study4 was revised and refined in subsequent waves based on changes in participant response patterns.