What question did this study set out to answer?

The study aims to evaluate the effectiveness of large language models in classifying neuropsychological test summaries to aid in the diagnosis and prognostication of idiopathic normal pressure hydrocephalus.

March 21, 2026Open Access

Large Language Models for Interpreting Standardized Neuropsychological Summaries in Idiopathic Normal Pressure Hydrocephalus: A Pilot Comparative Study

Key Points

The study aims to evaluate the effectiveness of large language models in classifying neuropsychological test summaries to aid in the diagnosis and prognostication of idiopathic normal pressure hydrocephalus.
Retrospective analysis of shunt-treated iNPH patients and neurological controls using neuropsychological test summaries.
Evaluation of three large language models (ChatGPT-5, Gemini 2.5 Flash, DeepSeek) under standardized zero-shot prompts.
Comparison of diagnostic and prognostic performance against established clinical diagnoses.
ChatGPT-5 achieved the highest diagnostic accuracy of 78% with an AUC of 0.84.
Prognostic accuracy for ChatGPT-5 was 83%, with positive predictive values ranging from 87% to 96%.
Calibration metrics indicated ChatGPT-5's effectiveness in predicting the likelihood of postoperative improvement.

Abstract

Diagnosis and prognostication in idiopathic normal pressure hydrocephalus (iNPH) remain challenging. This study examined whether large language models (LLMs) can classify neuropsychological test (NPT) summaries for identifying iNPH and exploring postoperative shunt responsiveness. This retrospective study included 42 shunt-treated iNPH patients and 53 neurological controls. De-identified NPT summaries with a standardized structure across cognitive domains were evaluated using three LLMs (ChatGPT-5, Gemini 2.5 Flash, DeepSeek) under a standardized zero-shot prompt. Each model generated categorical outputs (iNPH-compatible) and confidence scores (0-100). For analysis, responses labeled “Compatible” were considered AI-positive, whereas “Incompatible” or “Indeterminate” responses were treated as AI-negative; outputs with low confidence were conservatively handled as negative classifications. Diagnostic performance was compared with established clinical diagnoses. Prognostic performance was assessed for postoperative improvement, defined as ΔiNPHGS ≥1. Diagnostic performance showed the highest accuracy for ChatGPT-5 (78%, 95% CI 68.6-85.1), followed by Gemini 2.5 Flash (67%, 95% CI 57.4-75.9) and DeepSeek (63%, 95% CI 53.1-72.2). Corresponding AUC values were 0.84, 0.67, and 0.63. Calibration metrics for ChatGPT-5 included a Brier score of 0.18 and a calibration slope of 0.94. In the exploratory prognostic analysis conducted in the shunted iNPH cohort (n = 42), postoperative improvement occurred in 37 patients, whereas 5 patients were classified as non-responders. Prognostic accuracy was 83% (95% CI 69.8-92.5) for ChatGPT-5, 67% (95% CI 52.1-78.8) for DeepSeek, and 64% (95% CI 49.8-76.9) for Gemini 2.5 Flash. Positive predictive values ranged from 87% to 96%. When model confidence was analyzed as a continuous predictor, AUC values for prognostic discrimination were 0.69 for DeepSeek, 0.56 for ChatGPT-5, and 0.53 for Gemini 2.5 Flash. Decision-curve analysis indicated varying net benefit patterns across threshold probabilities. LLMs applied to standardized NPT summaries demonstrated measurable diagnostic signal and exploratory prognostic potential in iNPH. These findings support the feasibility of NPT-based LLM assistance, but larger, balanced, externally validated cohorts are required before clinical implementation.

Read Full Paperexternally

AI से पूछें

Bookmark

View Full Paper