Scenario-based evaluation of large language models for reference accuracy in dermatology: literature retrieval on latent tuberculosis in psoriasis patients on anti-IL-17/23 therapy

Key Points

• This research aims to assess the accuracy of large language models in retrieving dermatological literature related to latent tuberculosis in patients undergoing specific therapies.
• Evaluated LLM performance on reference accuracy in dermatology literature retrieval.
• Focused on scenarios involving latent tuberculosis in psoriasis patients on anti-IL-17/23 therapy.
• Identified risk of hallucination and the need for verification.
• LLM performance showed significant variability with high risk of inaccuracies or 'hallucinations'.
• Findings stress the importance of independent verification for reliable outcomes.
• Recommendations include testing advanced query techniques and hybrid systems for improved retrieval.