Key points are not available for this paper at this time.
Technological intervention to support care areas that some people may not have access to is of paramount importance to promote sustainable development of good health and wellbeing. This study aims to explore the linguistic similarities and differences between human professionals and Generative Artificial Intelligence (AI) conversational agents in therapeutic dialogues. Initially, the MISTRAL-7B Large Language Model (LLM) is instructed to generate responses to patient queries to form a synthetic equivalent to a publicly available psychology dataset. A large set of linguistic features (e.g., text metrics, lexical diversity and richness, readability scores, sentiment, emotions, and named entities) is extracted and studied from both the expert and synthetically-generated text. The results suggest a significantly richer vocabulary in humans than the LLM approach. Similarly, the use of sentiment was significantly different between the two, suggesting a difference in the supportive or objective language used and that synthetic linguistic expressions of emotion may differ from those expressed by an intelligent being. However, no statistical significance was observed between human professionals and AI in the use of function words, pronouns and several named entities; possibly reflecting an increased proficiency of LLMs in modelling some language patterns, even in a specialised context (i.e., therapy). However, current findings do not support the similarity in sentimental nuance and emotional expression, which limits the effectiveness of contemporary LLMs as standalone agents. Further development is needed towards clinically validated algorithms.
Bird et al. (Wed,) studied this question.