Procedures to Verify Legitimacy and Uniqueness of Responses in an Online Study with U.S. Young Gay and Bisexual Men Who Use Stimulants

In this brief report, we describe our procedures to determine the legitimacy and uniqueness of responses in an online study with young gay, bisexual, and other men who have sex with men in the U.S. Our verification procedures included automated methods (e.g., Qualtrics’ fraud detection, and knowledge, attention, and consistency-checks), manual methods (e.g., reviewing individuals’ open-ended descriptions of an image), and verification video-conference or phone calls. In total, 9321 individuals completed our eligibility screener and 2637 met eligibility criteria (28.3%). However, we could only ascertain legitimacy and uniqueness of 251 of these entries (9.5% of eligible individuals). Automated and manual methods flagged 68.4 and 9.4% of eligible entries as non-legitimate or duplicate, respectively; another 12.6% of eligible entries were excluded for not being able to confirm legitimacy in verification calls. Researchers should consider a range of automated and manual verification procedures to ensure data quality in internet-based studies.