Integrating tongue image and health examination records for MASLD risk prediction via multimodal deep learning

Metabolic dysfunction-associated steatotic liver disease (MASLD) is a growing global health concern, but its early detection remains limited by the cost and accessibility of conventional diagnostic tools. This study aimed to develop an interpretable multimodal deep learning framework for MASLD classification by integrating tongue images with routine health examination data. A total of 711 matched tongue images and health records were collected from Shanghai Changhai Hospital between June and December 2024. Residual Network-50 was used to extract image features, while a graph attention network modeled structured clinical data. An attention-based fusion module integrated the 2 modalities for classification. DeepMASLD achieved an accuracy of 92.04% and an F1 score of 0.91, outperforming the corresponding unimodal models. Triglycerides, alanine aminotransferase, and total cholesterol were among the most influential structured features, while class activation maps highlighted the posterior and lateral tongue regions. This study presents an accessible, interpretable multimodal framework integrating tongue images with clinical indicators for MASLD screening, achieving 92.04% accuracy and an F1 score of 0.91. In summary, DeepMASLD may support scalable early detection, pending multicenter and histopathological validation.