Testing the Waters: Behavior across Participant Pools

Key Points

• To evaluate the external validity of laboratory experiments using university student pools by comparing their behaviors to representative and online non-student populations.
• Administered a large-scale incentivized survey measuring economic behaviors across an undergraduate student population, a representative sample of the US population, and Amazon Mechanical Turk (MTurk) workers.
• Evaluated historical lab participation records to detect selection bias and conducted supplementary laboratory experiments to test for observer effects.
• Student behavior established bounds on behaviors observed in other populations, showing consistent correlation patterns across behavioral domains across all samples.
• Non-student participant samples exhibited higher levels of decision noise compared to the undergraduate student pool.
• Lab participants differed from non-lab participants on only a small subset of attributes, and follow-up lab experiments showed no evidence of observer effects.