What type of study is this?

This is a Quantitative Study study.

October 18, 2025Open Access

Shallow Robustness, Deep Vulnerabilities: Multi-Turn Evaluation of Medical LLMs

View Full Paper

Ask AI

Bookmark

Models show accuracy drops from 91.2% to 13.5% under multi-turn conditions, indicating severe vulnerabilities.
Evaluation framework distinguishes shallow and deep robustness, offering insights into model performance during complex interactions.
Controlled interventions reveal context-based inputs are more harmful than direct suggestions, compounding model vulnerabilities.
Comparative analysis shows performance variability among models, with some recovering better than others under repeated interventions.

No takes yet. Share an insight, caveat, or question.

Framework assesses accuracy in multi-turn medical consultations, highlighting vulnerabilities in large language models.

Manczak et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark