What question did this study set out to answer?

The research aims to demonstrate how large language models can simulate human surveys to interpret statutory terms more effectively.

May 26, 2026

The Ordinary Meaning Bot: Simulating Human Surveys with LLMs

Key Points

The research aims to demonstrate how large language models can simulate human surveys to interpret statutory terms more effectively.
Simulated a human survey using GPT-4o with AI agents resembling demographics of 2,835 participants from a previous study.
Compared response distributions from AI-generated data with human data using the Kolmogorov-Smirnov test for statistical significance.
Addressed issues of hallucinations, explainability, and reproducibility in LLM applications.
No statistically significant difference found in response distributions between AI and human data (p = 0.915).
Introduced an Ordinary Meaning Bot that provides practical and accurate statutory interpretation.
LLM-based simulations presented as a viable alternative to traditional methods, with fewer reliability concerns.

Abstract

This Comment shows how large language models (LLMs) can help courts discern the ordinary meaning of statutory terms. Instead of relying on expert-heavycorpus‑linguistic techniques (Gries, 2026), the author simulates a human survey with GPT‑4o. Demographically realistic AI agents replicate the 2,835 participants in Tobias 2020 study on vehicle and yield response distributions with no statistically significant difference from the human data (Kolmogorov-Smirnov p = 0.915). The paper addresses concerns about hallucinations, reproducibility, training-data contamination, and explainability, and introduces the locked‑prompt Ordinary Meaning Bot, arguing that LLM-based survey simulation is a practical, accurate alternative to dictionaries, intuition, or complex corpus analysis.

Bookmark

The Ordinary Meaning Bot: Simulating Human Surveys with LLMs

Key Points

Abstract

Cite This Study