XComposition: Multimodal Deep Learning Model to Measure Body Composition Using Chest Radiographs and Clinical Data

Background Body composition metrics such as visceral fat volume, subcutaneous fat volume, and skeletal muscle volume are important predictors of cardiovascular disease, diabetes, and cancer prognosis. Purpose We explore the use of deep learning to estimate body composition metrics from chest radiographs and a small set of easily obtainable clinical variables. Materials and Methods A retrospective cohort of patients with concurrent non-contrast abdominal CT’s and frontal chest radiographs within 3 months of each other was selected. A multitask, multimodal deep learning model utilizing chest radiographs and clinical variables (age, sex at birth, height and weight extracted from electronic medical records) was trained to estimate the body composition metrics. Reference standard was body composition, including subcutaneous fat volume, measured on CT. Results Our final cohort consisted of 1,118 patients (582 female and 538 male subjects) from 30 health systems across the United States with imaging performed 2010-2024. The mean age at imaging was 67 years (SD: 17), mean height was 1.67 meters (SD: 0.2), and mean weight was 78 kg (SD: 20). Average values for visceral fat, subcutaneous fat, and skeletal muscle indices were 59.39 cm2/m2 (SD: 39.26), 88.13 cm2/m2 (SD: 58.52) and 44.81 cm2/m2 (SD: 15.49). The best-performing model achieved a Pearson correlation of 0.85 (95% CI: 0.81–0.88) for subcutaneous fat volume, 0.76 (0.65–0.80) for visceral fat volume, and 0.58 (0.49–0.67) for skeletal muscle volume with the multimodal model outperforming unimodal models (p-value of 0.0001 for subcutaneous fat volume). Mean absolute errors of the best performing models for subcutaneous and visceral fat volumes were 1054 cm3/m2 and 667 cm3/m2, respectively. Conclusion We introduced a multimodal deep learning model leveraging chest radiographs to estimate body composition. Our model can facilitate large-scale studies by estimating body composition using a chest radiograph and commonly available clinical variables.