Key points are not available for this paper at this time.
High-quality experimental data is crucial for human-subject research. The increasing prevalence of online experimental studies has raised concerns regarding the quality and reliability of data collected in such environments. Recent evidence indicates that data quality varies across popular crowdsourcing platforms. We test if this also holds for less complex, easily comprehensible tasks. We find that compliance rates in a simple rule-following task are significantly lower in our Amazon Mechanical Turk sample compared to those from Prolific Academic and a German university lab. • Behavioral data quality varies across online crowdsourcing platforms. • Participants conduct a simple rule-following task, namely the coins-task. • We recruit participants on MTurk, Prolific and from a university lab database. • Data quality and rule compliance on MTurk is different compared to the other two. • Our findings align with other studies showing decreased data quality on MTurk.
Suri et al. (Fri,) studied this question.