Discovering natural kinds of robot sensory experiences in unstructured environments

We address the symbol grounding problem for robot perception through a data‐driven approach to deriving categories from robot sensor data. Unlike model‐based approaches, where human intuitive correspondences are sought between sensor readings and features of an environment (corners, doors, etc.), our method learns intrinsic categories (or natural kinds) from the raw data itself. We approximate a manifold underlying sensor data using Isomap nonlinear dimension reduction and apply Bayesian clustering (Gaussian mixture models) with model identification techniques to discover categories (or kinds). We demonstrate our method through the learning of sensory kinds from trials in various indoor and outdoor environments with different sensor modalities. Learned kinds are then used to classify new sensor data (out‐of‐sample readings). We present results indicating greater consistency in classifying sensor data employing mixture models in nonlinear low‐dimensional embeddings.