The sarcopenia artificial intelligence diagnostic decision support system (SAID DSS) – a multimodal deep learning model

Figures

• 

  Fig. 1. Diagrams of the different multimodal models’ approaches. (A) Voting model, (B) Decision-Fusion model, and (C) Feature-Fusion model. CNN: Convolutional Neural Network

• 

  Algorithm 1. Multimodal POCUS-based AI framework for sarcopenia diagnosis

• 

  Fig. 2. Receiver operating characteristic (ROC) curves and corresponding area under the curve (AUC) values for the multimodal convolutional neural network (CNN) models

• 

  Fig. 3. Methodology diagram of the final model. Diagram of the best-performing multimodal model. Clinical inputs include age, sex, SPPB score, height, weight, BMI, and number of falls. MLP: Multi-Layer Perceptron

• 

  Fig. 4. Spearman correlation between SHAP feature importance and Grad-CAM activation regions in thigh ultrasound images. Warm colors denote positive and cool colors negative correlations. The results highlight coherent associations between clinical features and image-based regions, reflecting physiologically meaningful patterns in sarcopenia detection

• 

  Fig. 5. Global SHAP summary plot for clinical feature importance. Each point represents a single patient, where color denotes feature value (red = high, blue = low) and the position along the X-axis indicates the feature’s impact on model output. BMI and age show the strongest influence on sarcopenia prediction, while SPPB contributes a mild protective effect toward non-sarcopenia classification

• 

  Fig. 6. Local SHAP waterfall plot for an individual patient. The plot illustrates how each feature shifts the model output from the baseline prediction (E[f(X)] = 0.623) to the final predicted value (f(x) = 0.19). BMI and age have the largest negative contributions toward sarcopenia classification, whereas height exerts a small positive effect, reflecting the clinical relevance of body composition and functional factors

• 

  Fig. 7. Cross-modal interpretability combining Grad-CAM and SHAP analyses for a representative sarcopenic patient. The Grad-CAM heatmap (left) highlights activation in the upper thigh muscle layer, reflecting the model’s focus on overall tissue density. The SHAP bar plot (right) shows that BMI and weight were the strongest contributors to the sarcopenia prediction, while age had a minor negative influence. Together, these results demonstrate coherent alignment between visual and clinical reasoning in the multimodal model’s decision process

• 

  Fig. 8. True Positive examples of sarcopenia detection with Grad-CAM visualizations. Each pair displays the original ultrasound image (left) and the corresponding Grad-CAM heatmap (right), highlighting the region’s most influential to the model’s prediction

• 

  Fig. 9. True Negative (TN) examples of correctly classified non-sarcopenic cases using Grad-CAM visualization. Each pair shows the original ultrasound image (left) and its corresponding heatmap (right), highlighting model attention patterns consistent with normal muscle structure

• 

  Fig. 10. False Positive (FP) examples showing non-sarcopenic cases misclassified as sarcopenic, with original ultrasound images (left) and corresponding Grad-CAM heatmaps (right). Model activations concentrate on non-diagnostic superficial regions, contributing to incorrect predictions

• 

  Fig. 11. False Negative (FN) examples of sarcopenic cases misclassified as non-sarcopenic. Each pair shows the ultrasound image (left) and corresponding Grad-CAM heatmap (right), illustrating instances where the model focused on superficial or non-diagnostic regions rather than deeper muscle structures