People are frequently asked to answer legally relevant questions on standardized forms that seek to elicit their judgments, beliefs, and behaviors. When the subject matter is complex, binary questions designed to ease administration or analysis may frustrate respondents, prompting them to second-guess the questions’ purpose and thereby answer inaccurately. We experimentally investigated whether providing respondents an opportunity to comment following a closed-form binary question affects accuracy. We presented a demographically representative sample of U.S. adults on CloudResearch (N = 1,693) with five richly described vignettes depicting different types of false or misleading speech. After each vignette, respondents judged whether or not the speech was a lie. In the primary No Frame treatment, respondents were randomly assigned either to a condition presenting only the closed-form binary question or to a condition presenting that question along with an opportunity to comment. In a secondary Moral Frame treatment, all respondents were first exposed to explicit moral evaluations of the speech acts before classifying them. Providing an opportunity to comment led to a small improvement in the accuracy of binary classifications in the No Frame conditions, but to no improvement in the Moral Frame conditions. These findings suggest that allowing space for elaboration can reduce distortions that arise when nuanced judgments are limited to binary formats. We discuss how even modest improvements in accuracy can have significant consequences where binary responses determine how individuals are treated, such as juror selection, insurance assessments, or eligibility for public or private benefits.
Stone et al. (Thu,) studied this question.