Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
December 4, 2025Journal of Korean Academy of Psychiatric and Mental Health NursingOpen Access

Verification of the Validity and Reliability of Therapeutic Communication Responses Generated by Large Language Models (LLMs): A Comparative Study of Prompt-Engineering Strategies

View Full Paper
Ask AI
Bookmark
Share

Authors

GKGeun Myun Kim

Discussion

Loading...

Member takes

Overview

Methodological design evaluated reliability and content validity of therapy responses in psychiatric nursing education, indicating prompt strategies matter.

Key Points

  • Role-prompt showed the highest content validity, followed by Chain-of-Thought and Zero-shot strategies.
  • Inter-rater agreement was low, with a Fleiss' κ of 0.07 and Krippendorff's ⍺ of 0.15, highlighting variability in expert assessments.
  • Semantic consistency scores were highest for Chain-of-Thought and Role-prompt methods, suggesting certain prompts enhance response quality.
  • Findings emphasize the importance of effective prompt-engineering strategies in optimizing therapeutic communication with large language models.

Cite This Study

Geun Myun Kim (2025) studied this question.

synapsesocial.com/papers/6930e8bdea1aef094cca326ahttps://doi.org/10.12934/jkpmhn.2025.34.s1.23
View Full Paper
Ask AI
Bookmark
Share

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs2024 · 380 citations
  2. 2Evaluation of large language models as a diagnostic tool for medical learners and clinicians using advanced prompting techniques2025 · 10 citations
  3. 3An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study2024 · 208 citations
  4. 4Simple Prompting Enhances ChatGPT’s Diagnostic Accuracy in Psychiatric Cases2025
  5. 5Evaluating Prompt Engineering Techniques for Accuracy and Confidence Elicitation in Medical LLMs2025 · 1 citations