Abstract Recent advancements in Large Language Models (LLMs), including OpenAI’s ChatGPT-4, Anthropic’s Claude 3, and Google’s Gemini Pro and Gemini Flash, have demonstrated exceptional linguistic fluency and contextual reasoning. However, certain behavioural patterns—especially those shaped by token distribution biases—remain underexplored. This study investigates a peculiar and reproducible phenomenon observed across state-of-the-art LLMs when prompted with the neutral instruction: “Pick a number between 1 and 50.”In over 800 automated, session-isolated trials, the number 27 appeared disproportionately as the first response, with an occurrence rate of >92% across models. This effect diminished dramatically (to < 3%) when the same prompt was repeated within a continued session context. The observed behaviour highlights the deterministic pseudo-randomness inherent in LLM outputs—apparent randomness that emerges from probabilistic token generation conditioned on pretraining data distributions. This pattern was consistently observed in several LLMs when initialized from a clean context. An automated pipeline was developed using Python and pyautogui, capturing and archiving screenshots for every output to ensure reproducibility. These findings suggest that LLMs inherit and amplify human-generated statistical quirks from their training corpora, reflecting latent biases even in prompts intended to elicit randomness. The study contributes to the expanding discourse on LLM interpretability, cognitive defaults, and bias detection, offering both a reproducible dataset and an open-source automation toolchain to encourage further exploration of emergent decision-making behaviours in generative models.
Shiva Nath (Wed,) studied this question.