A sentence is worth a thousand pictures: can large language models understand hum4n L4ngu4ge and the W0rld behind W0rds?

The current generation of large language models (LLMs) has been linked to claims about human-like linguistic performance, and their applications are hailed both as a step towards artificial general intelligence and as a major advance in understanding the cognitive and even neural basis of human language. To assess these claims, first, we analysed the contribution of LLMs as theoretically informative representations of a target cognitive system versus atheoretical mechanistic tools. Second, we evaluated the models' ability to see the bigger picture through top-down feedback from higher levels of processing, which requires grounding in previous expectations and past world experience. We hypothesize that since models lack grounded cognition, they cannot take advantage of these features and instead solely rely on fixed associations between represented words and word vectors. To assess this, we ran a novel leet task (l33t t4sk), which requires decoding sentences in which letters are systematically replaced by numbers. In line with our hypothesis, the results suggest that humans excel in this task, whereas models struggle. We interpret these results by identifying the key abilities that are still missing from the current state of development of these models, which require solutions that go beyond increased system scaling. This article is part of the theme issue 'World models in natural and artificial intelligence'.