Key points are not available for this paper at this time.
Headache disorders affect a large proportion of the global population, and Large Language Models (LLMs) show potential in the provision of online health information services, with their response accuracy and readability requiring verification. This cross-sectional study used an expert panel of five headache specialists and three clinical neuroscience pharmacists to evaluate the quality and readability of LLM(represented by GPT-5-main) generated responses to 53 headache-related public inquiries paired with responses from physicians and pharmacists from Chinese online health communities (OHC)-Haodf.com (Jan 2020 to Dec 2024) via six evaluation dimensions and the AlphaReadabilityChinese tool, with stratified analyses by inquiry difficulty (Level 1: simple; Level 2: complex). The results showed that LLM responses were significantly longer than responses from physicians and pharmacists ( P 0.05), and we recommend that Level 2 inquiries be subject to human review to optimize safety and relevance. Responses from physicians and pharmacists were significantly more readable than LLM across multiple metrics (all P <0.001). In conclusion, for headache-related enquiries, LLM responses were rated more favorably by experts across multiple dimensions within this dataset and can serve as valuable adjuncts to OHC, independently resolving simple enquiries. However, readability improvements are needed for direct public use, and review of physicians and pharmacists is required for complex cases.
Yang et al. (Fri,) studied this question.