What question did this study set out to answer?

The central aim is to enhance risk detection in medical chatbot answers using advanced language models.

April 1, 2026Open Access

TUSNLP at the NTCIR-18 MedNLP-CHAT Task: Utilization of External Medical Knowledge and Hybrid Approach of BERT and ChatGPT

Key Points

The central aim is to enhance risk detection in medical chatbot answers using advanced language models.
Developed model systems using BERT and ChatGPT.
Evaluated performance in detecting medical, legal, and ethical risks.
Implemented a hybrid approach combining both models.
ChatGPT outperformed in detecting medical risks.
BERT excelled in legal and ethical risk detection.
The hybrid model achieved the highest recall values across all risk types.

Abstract

We developed model systems for detecting medical, legal, and ethical risks in medical chatbot answers by using BERT and ChatGPT language models. The ChatGPT model system, which refers to external medical knowledge, performed best in detecting medical risk, while the BERT model system performed well in detecting legal and ethical risks. The hybrid model system reduces missed risks by combining the best of the BERT and ChatGPT model systems and has the best recall values for all risk determination models. This study demonstrates the usefulness of utilizing external medical knowledge and the effectiveness of the hybrid approach.

Bookmark

View Full Paper