Applying Direct Preference Optimization to Large Language Models for Suppressing Excessive Responses in Virtual Standardized Patient.

Medical interviews are a fundamental element of medical education and are an essential clinical skill thatstudents must acquire. Standardized patients (SPs), who imitate the behaviors of real patients, are important inthe training and examination of medical interviews. However, securing skilled SPs who can consistently followdetailed instructions and scenarios is both labor-intensive and costly. To address this problem, many studies haveproposed virtual standardized patients (VSPs), which instruct large language models (LLMs) to mimic SPs based ona given scenario. However, such straightforward approaches produce undesirable excessive responses. To addressthis problem, a VSP with mechanisms to suppress excessive responses has been proposed, although it suffers fromincreased response time because of its iterative self-reflection and revision processes. To overcome this limitation,this study proposes an LLM-based VSP that generates non-excessive responses at practical speed by training anLLM with synthetic medical interview dialogue data. We also propose a method to create synthetic dialogue databy simulating medical interviews between a conventional VSP with excessiveness suppression mechanisms and anLLM that replicates physicians’ questioning behavior. By using the stepwise excessiveness suppression mechanismof the conventional VSP, we construct a synthetic dialogue dataset consisting of triplets of a physician’s question, anexcessive response, and a non-excessive response. Using this dataset, we apply direct preference optimization to anLLM, thereby developing a VSP that directly generates non-excessive responses at practical speed without relying onexternal suppression mechanisms used in conventional approaches.