Cross‐Modal Activation, Integration, and Validation When Reading Illustrated Texts: Evidence From Reading Times and Eye‐Tracking

The experiments investigated cognitive processes involved in reading illustrated text, focusing on cross‐modal activation, integration, and validation between text and picture. Using an adapted contradiction paradigm, both studies manipulated, within a single illustrated narrative, whether a previously presented picture was consistent or inconsistent with a later target sentence. Experiment 1 ( N = 285) employed sentence‐by‐sentence presentation and key‐press reading times, conceptually replicating earlier findings. Experiment 2 ( N = 143) extended the paradigm to a more naturalistic context, presenting short paragraphs on each page and using eye‐tracking. Across both experiments, inconsistencies led to longer processing times: Experiment 1 revealed prolonged reading times for the spillover sentence; Experiment 2 showed longer processing of the target sentence. Increased gaze transitions between the target sentence and the picture presented on the same page were observed. These results provide preliminary evidence that readers engage in cross‐modal activation, integration, and validation processes when reading illustrated texts.