Answers are grounded in this paper and cite their sources.
Learning dexterous manipulation from human demonstrations requires records of physical interactions that video alone cannot fully capture. Here we present tactile-motion capture (TaMoCap), a portable physical interaction tracking system that records multimodal tactile–posture information during natural human demonstrations, reconstructs physical interaction states of human-object interaction, and converts them into structured semantic annotations. The system integrates a smart textile glove with distributed three-axis force, triboelectric and strain sensors. The synchronized multimodal sensing streams are processed to reconstruct three-axis contact forces, strain-derived joint-angle estimates, and triboelectricity-based contact-transition states, yielding physically interpretable representations that characterize sustained loading, transient interaction events, and hand posture during natural manipulation. A tactile semantic translation framework integrates synchronized visual observations, reconstructed physical interaction states, and task context to generate action annotations, which are verified against multimodal evidence for physical consistency and source traceability. In a descriptive evaluation of four selected massage cases, the full visual–tactile–semantic input achieves a mean composite score of 94.75/100 in model-based assessment, 33.5 points higher than the video-only input. In a long-horizon noodle-cooking demonstration, TaMoCap preserves temporal alignment between multimodal sensing evidence and structured semantic annotations throughout the action sequence. TaMoCap provides a systematic approach for constructing physically grounded human demonstration records for tactile-enhanced embodied agents and robotic manipulation systems.
No takes yet. Share an insight, caveat, or question.
Fan et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: