Multimodal Measurements Fusion for Surface Material Categorization

Sound and acceleration measurements are two classes of sensing modalities which frequently occur in surface material categorization. Their fusion problem is extremely important in many practical scenarios, since they provide different properties about materials. In this paper, we investigate the multimodal measurements fusion categorization problem exhibiting nontrivial challenges that there does not exist sample-to-sample pairing relation between sound and acceleration measurements. To this end, we design a dictionary learning model that can simultaneously learn the projection subspace and the latent common dictionary for the different measurements. Furthermore, an optimization algorithm is developed to effectively solve the common dictionary learning problem. Based on the obtained solutions, the fusion categorization algorithm can be easily developed. Finally, we perform experimental validations on the publicly available data set to show the effectiveness of the proposed method.