Can artificial intelligence reliably support MRI-based grading of lumbar paraspinal muscle fat infiltration? A multispecialty comparison

Fat infiltration of the lumbar paraspinal muscles is an important imaging biomarker associated with spinal degeneration and functional impairment. However, semiquantitative MRI-based grading systems are inherently observer-dependent, and the reliability of artificial intelligence (AI) systems in such tasks remains unclear. This retrospective observational study included 130 patients who underwent lumbar spine MRI. Fat infiltration was graded using the Kjaer and Suzuki classifications by two neurosurgeons, three radiologists, three physical medicine and rehabilitation specialists, and two large language model–based AI systems (ChatGPT and Gemini). Interobserver agreement was assessed using weighted Cohen’s kappa and intraclass correlation coefficients (ICC). Differences in grading distributions were analyzed using the Friedman test with post-hoc comparisons. Agreement between AI systems was poor (κ = −0.09), indicating below-chance concordance. Interobserver reliability varied across specialties, with substantial agreement among neurosurgeons (κ = 0.64), moderate agreement among physical medicine and rehabilitation specialists (κ = 0.51), and low agreement among radiologists (κ = 0.20). Agreement between AI models and the human consensus was low for both ChatGPT (κ = 0.14) and Gemini (κ = 0.12). In contrast, specialty-specific consensus scores demonstrated strong concordance (κ = 0.71–0.77). Grading distributions differed significantly across evaluator groups ( p = 0.0035). Consensus-based grading of lumbar paraspinal muscle fat infiltration demonstrates high reliability across specialties, despite variability at the individual level. Current large language model–based AI systems show limited agreement with human experts and are not yet sufficiently reliable for observer-dependent MRI-based grading tasks. At present, AI should be considered a supportive tool rather than a substitute in clinical spine imaging.