Personality without a person: measuring personality in generative AI

Personality describes relatively stable styles of thinking, feeling, and acting when interacting with the physical and social environment. The stability and reliability of these individual styles plays an important role in our daily lives as we team with various individuals at work and home. Generative AI agents (LLMs) are new additions to our lives, quickly assuming roles that might otherwise have been given to humans though there is little known about the stability and reliability of LLM “thinking” styles or actions. We have been investigating whether LLMs can be thought of as having personality out of the box along with the creation and measurement of AI agents with distinct and reliable personalities. In this study four LLMs were assessed with both self-report and newer gamified behavioral measures of personality under an Out-of-the-Box condition and with three prompt-based personas. Contrary to previous studies, LLMs showed very high test-retest reliability. Personality scores varied predictably following persona prompting, while agreement between assessment types varied across models and persona conditions.