Using Artificial Neural Networks to Relate External Sensory Features to Internal Decisional Evidence

Abstract All theories of perceptual decision-making postulate that external sensory information is transformed into the internal evidence that is used to judge the identity of the stimulus. However, the nature of this external-to-internal transformation is generally unknown. In two experiments, we examined how a particular stimulus feature—orientation—is transformed into internal evidence. Subjects judged whether Gabors were tilted clockwise or counterclockwise. The results of Experiment 1 demonstrated that increasing the stimulus tilt in fine-scale increments resulted in a linear increase in sensitivity. However, the results of Experiment 2 demonstrated that increasing the stimulus tilt in coarse-scale increments had little effect on sensitivity, suggesting a highly non-linear transformation. Critically, artificial neural networks (ANNs) trained on the orientation task reproduced the empirical results, providing a framework for examining this external-to-internal transformation. The ANNs’ internal activations revealed that fine-scale increments in tilt magnitude results in increasingly greater discriminability between the stimulus categories, but the degree of discriminability does not increase further after tilt magnitude becomes sufficiently large. Taken together, these results begin to reveal how external sensory information is transformed into the internal evidence that is used to judge the identity of a stimulus and suggest that ANNs could serve as a platform for understanding the mechanism underlying this critical transformation.