Towards simultaneous categorization and mapping among multimodalities based on subjective consistency

This paper proposes a method for acquiring categories in one modality and mappings between these categories and those in other modalities. Subjective consistency through multimodal mappings is introduced to judge to what extent a perceived signal and inferred ones from other modalities are reliable for categorization and mapping. Based on the proposed method, a simulated infant robot learns categories and mappings by using not only statistics on one perceptual modality but also mappings among the categories in other modalities. The proposed method enables partly simultaneous categorization and mappings.