Large Language Models (LLMs) have revolutionized natural language processing, but they face a critical limitation: the context window. Despite recent advances that have expanded context windows to millions of tokens, models still suffer from “context rot” — a systematic degradation in performance as input length increases. This paper explores Recursive Language Models (RLMs), a novel inference paradigm inspired by level-of-detail (LOD) and streaming techniques from open-world video games such as Minecraft, Grand Theft Auto, Assassin’s Creed, Red Dead Redemption, and Cyberpunk 2077. RLMs conceptualize large prompts as external environments that models can programmatically explore through recursive sub-queries, enabling effective processing of inputs up to 100× beyond base model context windows. We examine the technical parallels between game rendering optimization and context management, analyze empirical results showing that RLMs outperform vanilla frontier LLMs by up to 114% on information-dense tasks, and discuss the implications for future AI systems capable of handling arbitrarily long contexts.
Zen Revista (Sat,) studied this question.