Prompt optimization and multidimensional quality assessment for Chinese large language model-generated diagnostic impressions from liver MRI findings: external-center and cross-disease validation

To develop and validate a structured prompt optimization strategy and a Multi-Dimensional Quality Assessment (MDQA) framework for evaluating LLM-generated diagnostic impressions from imaging fundings text. This retrospective study included 15,127 primary-center liver MRI reports. Imaging findings were provided to Chinese LLMs, whereas consensus diagnostic conclusions confirmed by two associate-senior or senior radiologists were withheld for evaluation. The MDQA framework assessed Semantic Coherence, Diagnostic Correctness, Clinical Prioritization Alignment, Top-1 matching, and a composite score. Prompt composition, example number, and model type were evaluated across Kimi-K2, DeepSeek-V3, ByteDance-Seed-OSS-36B-Instruct, and Qwen3-235B-A22B-Instruct-2507. Cross-center validation used 1,009 external liver MRI reports, and cross-disease validation used 1,692 pulmonary thin-section CT reports. Radiologist ratings were used for consistency assessment. Structured prompts improved diagnostic correctness, prioritization, and overall MDQA scores compared with prompts containing only role definition or basic task instructions. Increasing example number mainly improved semantic coherence and Top-1 matching; the composite score peaked at 0.7696 for Kimi-K2 with Prompt 11 and 0.7613 for DeepSeek-V3 with Prompt 10. In the external cohort, locally adapted Prompt 11 V3 achieved the highest MDQA scores for Kimi-K2 and DeepSeek-V3. In lung disease validation, the MDQA score increased from 0.4773 ± 0.1701 with Prompt 0 to 0.7109 ± 0.1676 with Prompt 10. Radiologist ratings followed similar trends. Structured prompts and MDQA-based evaluation may improve the consistency and report-level quality of LLM-generated diagnostic impressions from radiology findings.