Tactile-FOAA: a perceptual modality fusion framework for bionic optoelectronic tactile systems

Tactile perception plays an irreplaceable role in robotic fine manipulation and object recognition. Aiming at the significant differences between dynamic and static tactile signals in temporal scales, spectral characteristics, and response characteristics, this paper introduces flattened outer arithmetic attention (FOAA) into tactile multimodal fusion and proposes a perceptual modality fusion framework for bionic optoelectronic tactile systems, namely tactile-FOAA (T-FOAA). The framework consists of a fiber Bragg grating (FBG) optoelectronic tactile acquisition system, a dynamic-static dual-parameter demodulation module, and a multi-view arithmetic fusion model based on FOAA, which can synchronously acquire two types of heterogeneous tactile signals: static stress and dynamic vibration. Furthermore, in the fusion stage, T-FOAA incorporates four outer arithmetic operations, namely outer addition (OA), outer subtraction (OS), outer product (OP), and outer division (OD), to introduce multiple arithmetic interaction forms between dynamic and static modalities, thereby enriching the physically motivated cross-modal interactions of the fused representation. Based on a self-constructed tactile multimodal dataset containing 16 hardness-texture combinations with a total of 14,400 samples, experimental results show that T-FOAA achieves a mean recognition accuracy of 94.92%±0.35% in tactile multimodal classification tasks, outperforming the evaluated multimodal fusion baselines under the same experimental settings. This study provides a lightweight, high-classification-accuracy, compact and efficient framework for tactile multimodal perception.