The integration of Large Language Models (LLMs) into mental health creates a novel paradigm of human-AI interaction, where users engage artificial agents as social and therapeutic partners. This review synthesizes evidence from studies evaluating a range of LLMs, including GPT-3.5, GPT-4, Gemini, and Claude, reflecting the state of the art up to early 2025. This systematic review with narrative synthesis synthesizes current evidence to explore the psychological dynamics of these interactions. We investigate how LLMs, as artificial humans, foster perceptions of empathy and therapeutic alliance, and how user behaviors like anthropomorphism and prompt engineering co-create the interaction. Our analysis reveals a central tension: while LLMs effectively simulate empathetic dialogue, leading to positive user experiences and symptom reduction, this compelling social presence masks a critical accountability vacuum. Users risk forming an illusive alliance—a perceived bond with an entity lacking genuine understanding or clinical responsibility. This empathy-accountability gap highlights profound risks, including over-trust and inadequate crisis response, arising from the mismatch between human social perception and AI's operational reality. We argue that the future of LLMs in mental health lies not in perfecting autonomous artificial therapists, but in designing augmented intelligence systems that explicitly manage these human-AI relational dynamics within ethical frameworks. The findings urge a shift in focus from clinical efficacy alone to the psychological intricacies of building trust, managing expectations, and ensuring safety in therapeutic relationships with artificial agents. • Proposes the "empathy-accountability gap" as a novel framework for analyzing human-AI therapeutic relationships. • LLMs foster an "illusive alliance" through simulated empathy but fail in clinical safety during crises. • Anthropomorphism and user prompt engineering co-create both engagement and risk with artificial therapists. • Argues for augmented intelligence—not autonomous agents—as the ethical path forward for mental health AI. (86 chars)
Azeem et al. (Sun,) studied this question.