In the quest to enhance medical consultation, our study introduces AI4Doctor, a sophisticated large-language model (LLM) tailored for the clinical domain. At the heart of AI4Doctor is an innovative integration strategy that synergizes distilled data extracted from electronic medical records (EMR) with empirical insights gathered from practicing physicians during the supervised fine-tuning. Although existing platforms offer informative responses, they fall short of replicating the nuanced decision-making processes of medical professionals, particularly in complex, integrative diagnostic scenarios. Motivated by the need to create a realistic medical practice environment, we propose that a combination of direct knowledge transfer from seasoned doctors and the strategic use of EMR can augment the abilities of LLM, enabling it to more closely mimic the clinical acumen of healthcare practitioners. To navigate the complexities of merging diverse instructional sources, we employ a curriculum learning approach during the fine-tuning process. Moreover, we advance our model’s performance by developing a reward system that incentivizes the alignment of the LLM’s outputs with the valuable attributes inherent in both doctors’ expertise, including diagnostic priors, risk thresholds, and heuristic saliencies accumulated from practice and EMR data. This is achieved through a novel reinforcement-learning approach. Besides, we introduce a new benchmark involving a comparative evaluation. We utilize a subjective evaluation system wherein experts critically assess the responses from a professional perspective as well. Our research underscores the potential of this hybrid model to serve as a robust tool in medical consultations, bridging the gap between artificial intelligence and real-world clinical practice.
Zhuang et al. (Fri,) studied this question.