Properties of Person Descriptors in the Natural German Language: A Preregistered Replication and Extension

Understanding language is crucial to psychological research, as it is the basis foralmost all psychological measurements. Building on previous work, we conducted a preregistered replication study, analyzing 874 person-descriptive terms generated by 187 participants using a free response format. These terms were rated on social desirability, observability, importance, abstractness, base rate, and stability by approximately human 15 raters each (n = 456). Key findings were replicated, including a bimodal distribution of social desirability, a greater number of negative vs. positive terms, and an U-shaped relationship between importance and social desirability. Furthermore, we show that human ratings of social desirability can be closely approximated by synthetic ratings derived from either a fine-tuned encoder model or the general-purpose transformer model GPT-4o. However, when investigating GPT-4o's performance in approximating human ratings of the other properties, it was mediocre at best, and its performance in approximating abstractness was poor. This suggests that there is still need for improvement in AI-applications for natural language processing.