CPAD: Cross-Modal Physics-Aware Disentanglement for Robust Acoustic Sensing in Gesture Recognition

In the field of ubiquitous acoustic sensing, gesture recognition faces severe performance degradation when generalizing to novel classes under heterogeneous sensor variations and data imbalance. Deep learning models often struggle to distinguish motion-relevant acoustic patterns from device/domain-related nuisance factors. We propose cross-modal physics-aware disentanglement (CPAD), a framework that leverages large language model (LLM)-generated structured semantic descriptions as topology-shaping anchors, rather than explicit physical simulation, to encourage the separation of motion-related features from device/domain-related nuisance factors. To overcome the capacity saturation caused by narrow bottlenecks, CPAD employs an auxiliary projection head paradigm. The dual-branch orthogonality and physics-alignment constraints are utilized strictly as regularizers during pretraining. During inference, these projection heads are discarded, and metric matching is performed directly on the regularized backbone features. Experiments on the ISAC-Gesture dataset show that CPAD achieves a 7.28% absolute improvement over a strictly fair baseline under the dataset-defined unseen-device protocol, reaching 76.25% average accuracy in five-shot scenarios on the available Xiaomi target-device split. For deployment-oriented validation, we report an Android profiling protocol with explicitly stated measurement scopes, covering ONNX export, INT8 quantization, isolated forward-pass latency, few-shot timing, and BatteryManager-based power estimation. INT8 achieves approximately 2.44–2.46×isolated ONNX backbone forward-pass acceleration and reduces the model footprint from 44.7 to 11.2 MB. On vivo X200 Pro mini (V2419A), 25 independent 180-s sustained neural-inference runs show lower mean latency (from 32.78 to 16.89 ms), higher throughput (from 30.33 to 58.82 infer/s), lower average whole-device power estimate (from 6448.74 to 4711.42 mW), and lower normalized per-inference energy (from 212.60 to 80.10 mJ). These results indicate that LLM-structured semantic descriptions can serve as topology-shaping anchors to mitigate sensor-induced degradation under the evaluated acoustic sensing protocol.