Can Large Language Models Exhibit Cognitive and Affective Empathy as Humans?

Empathy, a critical component of human-social interaction, has become a core concern in human-computer interaction. This study examines whether current large language models (LLMs) can exhibit empathy in both cognitive and affective dimensions, akin to humans. We proposed a novel paradigm for LLMs' evaluation based on the standardized questionnaires. Four main experiments were reported on LLMs' empathy abilities. Specifically, GPT-4 and Llama3 were tested as indexed by the Interpersonal Reactivity Index (IRI) and the Basic Empathy Scale (BES). Two levels of evaluations were conducted to investigate whether the structural validity of the questionnaire in LLMs is aligned with humans and to compare the LLMs’ empathy abilities with humans directly. GPT-4 showed identical empathy dimensions to humans while exhibiting significantly lower empathy abilities in both cognitive and affective dimensions. The completely different empathy ability was more evident in Llama3 by showing its failure to exhibit the same empathy dimensions as we humans. All these findings indicate that LLMs cannot convey empathy ability as we humans currently. This highlights the need for further development and fine-tuning of LLMs to enhance their empathy abilities. In addition, the way to prompt LLMs to simulate diverse LLMs-based participants was discussed as well as the sampling strategy.