Developing and validating the attitudes toward large language models scale

With large language models (LLMs) now embedded in a wide range of everyday technologies, there is increasing interest in accurately measuring public attitudes toward these systems. General AI attitude instruments may not fully capture attitudes specific to LLMs. To address this gap, we adapted the 5-item Attitudes Toward Artificial Intelligence (ATAI) scale, a dual-component scale measuring AI Fear and AI Acceptance, and developed two new measures: the Attitudes Toward General LLMs (AT-GLLM) and Attitudes Toward Primary LLM (AT-PLLM) scales, assessing general attitudes and attitudes toward the LLM each respondent uses most. Both instruments were designed specifically for the LLM context and validated in a UK sample of 526 adults (ages 18-45). Psychometric analyses supported a two-factor structure, strong measurement invariance across gender, and acceptable internal consistency across the four subscales (Cronbach’s α = 0.742–0.786; McDonald’s ω = 0.747–0.793). Both scales demonstrated convergent and discriminant validity. These measures offer more precise tools for assessing attitudes toward LLMs than general AI attitude scales and can be used in research and applied settings, enhancing evaluations of user experience and supporting the monitoring of public engagement with this evolving technology.