Material Classification under Natural Illumination Using Reflectance Maps

Research on visual material recognition has traditionally been based on texture analysis. Whereas older work has focused on uncluttered scenes, more recent contributions allowed for material recognition 'in the wild'. Quite some objects have untextured surfaces, however. Especially man-made examples are legion. The most obvious cue to use in such cases would be reflection information. Yet, methods to that effect are lacking. It is clear that more than an estimate of a scalar albedo is needed, and a more complete reflectance model has to be derived. Rather than using an extensive lab setup, we propose a system that only requires the 3D shape of objects and a regular, commercial camera to capture their appearance in a single image, to perform material classification under unknown illumination. To this end, we rely on a Gaussian Process Latent Variable Model (GPLVM) with a discriminative prior to learn a low-dimensional manifold suitable for material classification of reflectance maps, i.e. from a 2D image of a singlematerial sphere under natural illumination. We evaluated our method based on experiments generated from synthetic and real-life data. Although recognizing materials without texture (or object recognition) is not a trivial problem, our method achieves about 75% recognition accuracy, about 27% higher than human performance.