Cross-linguistic differences in responses to patient queries on osteonecrosis of the femoral head between English and Chinese-oriented large language models: a semantically matched bilingual cross-sectional study

Background Large language models (LLMs) are capable of providing health information and consultation to patients with osteonecrosis of the femoral head (ONFH). However, owing to imbalances in the linguistic distribution of their training corpora, the quality, readability, and safety profiles of outputs generated by the same model may vary across language settings. Objective This study aimed to compare bilingual responses to ONFH patient queries generated by six state-of-the-art flagship LLMs, comprising three English-oriented models (ChatGPT-5.6-Sol, Gemini-3.1-Pro, and Claude-Fable-5) and three Chinese-oriented models (DeepSeek-V4-Pro, Qwen3.7-Max, and Doubao-Seed-2.1-Pro), and to multidimensionally evaluate their cross-linguistic capabilities for generating medical information in terms of information quality, actionability, clinical accuracy, safety risk, hallucination, and readability. Methods A question set comprising 36 semantically distinct patient queries regarding ONFH was developed, yielding 72 semantically matched English–Chinese paired prompts. Six state-of-the-art flagship LLMs were evaluated. Information quality was assessed using the Global Quality Score (GQS), DISCERN, the Quality Assessment of Medical Artificial Intelligence (QAMAI), and the 20-item Ensuring Quality Information for Patients (EQIP-20). Actionability was assessed using the Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P), while clinical accuracy was evaluated using a 5-point Likert scale. Safety risk was assessed using a graded classification framework, and prespecified criteria were established for identifying hallucinations. Readability was evaluated using language-specific readability metrics for English and Chinese responses. Statistical analyses included the Friedman test, paired Wilcoxon signed-rank test, and Spearman correlation analysis. Results A total of 432 responses generated by LLMs were included in this study, and good-to-excellent inter-rater reliability was observed for all evaluation indicators. Significant between-model differences in actionability and clinical accuracy were identified among the six models under both English and Chinese response conditions (all p < 0.05). Furthermore, clinical accuracy exhibited high cross-linguistic stability. ChatGPT-5.6-Sol showed high actionability and a relatively low frequency of Level 1 minor safety-related omissions under the English-language condition, whereas switching to Chinese was accompanied by reduced actionability and an increased frequency of Level 1 omissions. All observed nonzero safety events were Level 1 minor omissions; no Level 2 or Level 3 unsafe responses were identified in either language condition. Responses generated by all six models in both languages failed to reach the prespecified readability threshold for patient education. Conclusion The six LLMs showed significant inter-model differences in the actionability and clinical accuracy of responses to ONFH-related patient health queries in both English and Chinese settings, while clinical accuracy demonstrated favorable cross-linguistic stability. Notably, none of the six-model outputs met the applicable readability threshold for patient-targeted health education under either language condition. In clinical practice, model selection, prompting-language effects and hallucination-related risks should be comprehensively considered when deploying LLMs to deliver health information for patients with ONFH. At the present stage, LLM-generated content intended for direct patient-facing education requires cautious evaluation and human oversight.