Unsupervised Learning of Invariant Representations in Hierarchical Architectures

The present phase of Machine Learning is characterized by supervised learning relying on large sets of labeled examples (n \ \). The next is likely to focus on algorithms capable of learning from very few examples (n \ 1), like humans seem able to do. We propose an to this problem and describe the underlying theory, based on the, automatic learning of a ``good'' representation for supervised, characterized by small sample complexity (n). We consider the case visual object recognition though the theory applies to other domains. The point is the conjecture, proved in specific cases, that image which are invariant to translations, scaling and other can considerably reduce the sample complexity of learning. We that an invariant and unique (discriminative) signature can be computed each image patch, I, in terms of empirical distributions of the-products between I and a set of templates stored during unsupervised. A module performing filtering and pooling, like the simple and cells described by Hubel and Wiesel, can compute such estimates. architectures consisting of this basic Hubel-Wiesel moduli inherit properties of invariance, stability, and discriminability while capturing compositional organization of the visual world in terms of wholes and. The theory extends existing deep learning convolutional architectures image and speech recognition. It also suggests that the main computational of the ventral stream of visual cortex is to provide a hierarchical of new objects/images which is invariant to transformations, , and discriminative for recognition---and that this representation may continuously learned in an unsupervised way during development and visual.