Robot Multimodal Object Perception and Recognition: Synthetic Maturation of Sensorimotor Learning in Embodied Systems

It is known that during early infancy, humans experience many physical and cognitive changes that shape their learning and refine their understanding of objects in the world. With the extended arm being one of the very first objects they familiarize, infants undergo a series of developmental stages that progressively facilitate physical interactions, enrich sensory information, and develop the skills to learn and recognize. Drawing inspiration from infancy, this article deals with the modeling of an open-ended learning mechanism for embodied agents that considers the cumulative and increasing complexity of physical interactions with the world. The proposed system achieves object perception and recognition as the agent (i.e., a humanoid robot) matures, experiences changes to its visual capabilities, develops sensorimotor control, and interacts with objects within its reach. The reported findings demonstrate the critical role of developing vision on the effectiveness of object learning and recognition and the importance of reaching and grasping in solving visually elicited ambiguities. Impediments caused by the interdependency of parallel components responsible for the agent's physical and cognitive functionalities are exposed, demonstrating an interesting phase transition in utilizing object perceptions for recognition.