What question did this study set out to answer?

This research aims to understand the differences between contextual computation in vision-language models and human perception, particularly regarding cognitive biases and perceptual illusions.

June 19, 2026Open Access

Vision-Language Models Assimilate Where Humans Contrast: A Cross-Architecture Signature of Contextual Computation

Key Points

This research aims to understand the differences between contextual computation in vision-language models and human perception, particularly regarding cognitive biases and perceptual illusions.
Utilized vision-language models as a no-retina visual substrate to assess contextual modulation.
Implemented a gate-controlled, prior-orthogonalised forced-choice readout to measure perceptual dimensions of size and brightness.
Compared responses across three distinct vision encoders to evaluate graded dose-responses of perceptual illusions.
VLMs showed assimilation effects, modulating targets toward their surroundings, differing from humans who exhibit contrast effects.
Replicated findings across three architectures with each slope’s 95% confidence interval excluding zero, indicating significant effects.
Evidence suggests that VLMs do not simply mimic human perception based on training data, pointing to fundamentally different contextual processing.

Abstract

Which “cognitive biases” and “perceptual illusions” are necessary signatures of a bounded decision architecture, and which are mere artefacts? Behavioural data alone cannot say, because the latent decision variable must be inferred from aggregated choices. The per-token competing-routes (log-probability) substrate of a language model makes that latent observable, and can therefore act as an instrument that sorts a candidate phenomenon into phantom (an observer-aggregation artefact), architecture-forced (a real bounded-commitment signature), or substrate accident. This paper states that sorting criterion and demonstrates it in perception, using vision-language models (VLMs) as a no-retina visual substrate. With a gate-controlled, prior-orthogonalised forced-choice readout, classical inferential illusions recur as contextual modulation with a measurable commitment signature — but with a substrate-specific sign: VLMs modulate a target toward its surround (assimilation) where human perception modulates it away (contrast). The effect is a graded dose-response that replicates across three architecturally distinct vision encoders (a native dynamic-resolution ViT, a SigLIP encoder, and a CLIP encoder) and across two perceptual dimensions — size (Ebbinghaus, Delboeuf) and brightness (simultaneous-contrast) — with every one of nine dose-response slopes’ 95% confidence interval excluding zero. The opposite sign is strong evidence against the simplest account — that VLMs imitate human illusion reports absorbed from text — and is consistent with a systematically different contextual computation; whether that sign is forced by the architecture or learned from training-data statistics is left as the live open question, with the cross-corpus convergence (the same sign across three models trained on different data) weighing against a dataset-specific account. Companion papers in the series develop the cognitive and self-knowledge faces of the same “biases-as-architecture” claim, the social-friction worked cases, and the competing-routes measurement-model programme. Prepared for submission to Transactions on Machine Learning Research (TMLR). Data and code. The stimulus generators, evaluation notebooks, and per-token log-probability outputs are available from the author.

Read Full Paperexternally

问 AI

Bookmark

View Full Paper