Semilinear Predictability Minimization Produces Well-Known Feature Detectors

Predictability minimization (PM—Schmidhuber 1992) exhibits various intuitive and theoretical advantages over many other methods for unsupervised redundancy reduction. So far, however, there have not been any serious practical applications of PM. In this paper, we apply semilinear PM to static real world images and find that without a teacher and without any significant preprocessing, the system automatically learns to generate distributed representations based on well-known feature detectors, such as orientation-sensitive edge detectors and off-center–on-surround detectors, thus extracting simple features related to those considered useful for image preprocessing and compression.