Do people use counterexamples or subjective probabilities in uncertain conditional reasoning? Investigating human reasoning with an open response format

Probabilistic approaches to human reasoning argue that all reasoning processes are inherently probabilistic and based on degrees of belief. To show this, participants are typically presented with scaled response formats, in which they can indicate how strongly they belief in a conclusion. However, the cognitive mechanisms behind uncertain reasoning remain unclear. How do people assess degrees of belief? This paper addresses this question by employing an open response format, allowing participants to decide on their own how to solve a reasoning problem. Thus, they could formulate their own conclusions and respond as spontaneously as possible without any constraints. Our findings reveal that participants generate more certain conclusions than is suggested by probabilistic theories of reasoning. Furthermore, counterexamples played a significant role, but in a more complex manner than previously assumed in the literature. Participants also gave uncertain conclusions based on probabilities, yet only for a subclass of inferences. We conclude that an open response format can provide valuable insights into how people naturally reason when not constrained by a particular research methodology.