Fusion VLM-Gait: RGB-D fusion in vision-language models for explainable multi-task Parkinsonian gait analysis

Parkinson’s disease (PD) gait analysis requires the identification of both subtle local motion abnormalities and broader whole-body movement changes. However, many existing approaches still rely on single-modality sensing or conventional multimodal fusion strategies that do not explicitly preserve these complementary cues, and few provide a contactless, multi-task, interpretable RGB-D vision-language pipeline for auxiliary gait assessment in clinically relevant settings. In this work, we present FusionVLM-Gait, a contactless RGB-D vision-language framework for multi-task Parkinsonian gait analysis. The method uses independent YOLOv11-based RGB and depth encoders, followed by a Multi-Scale Local-Global Extraction (MLGE) module that separates local limb-level motion information from global posture-level gait information before multimodal integration. These representations are then combined through a Cross-Spatial Neck Fusion (CSNF) mechanism designed to improve hierarchical cross-modal alignment and symptom-aware feature fusion. The fused representation supports classification, detection, and symptom-oriented prediction, which are subsequently translated into human-readable descriptions by a frozen large language model. Results on simulated Parkinsonian gait data indicated that the proposed local/global decomposition and cross-modal fusion design improved representation quality and interpretability relative to unimodal and conventional fusion baselines. This study demonstrates the feasibility of explainable, contactless RGB-D gait assessment using a vision-language framework.