Visual Maze Solving by Multimodal Large Language Models

We conducted an exploratory evaluation of multimodal LLM performance on a maze-solving and path-drawing task. The task required the model to find and draw, on a supplied image of a maze, a path from the starting point to the exit without crossing the walls. We tested proprietary models Grok, Gemini-3, ChatGPT-4o, and ChatGPT-5.2 on 20 mazes (10 with a path from the start to the exit and 10 without one). Each problem was presented five times in random order. We also tested the effect of language by presenting separate prompts in English and Croatian, resulting in a total of 200 trials per model (100 trials per language). We found that only ChatGPT-5.2 consistently produced correct answers in both verbal and visual modalities. We further examined drawing errors made by other models, such as visual hallucinations (inventing a new, unrelated maze), failing to connect the starting point to the exit, crossing the walls, and walking on the walls. We also conducted an exploratory study on the LLMs’ metacognitive judgments about their performance.