Efficient retrieval of deformable shape classes using local self-similarities

We present an efficient object retrieval system based on the identification of abstract deformable `shape' classes using the self-similarity descriptor of Shechtman and Irani. Given a user-specified query object, we retrieve other images which share a common `shape' even if their appearance differs greatly in terms of colour, texture, edges and other common photometric properties. In order to use the self-similarity descriptor for efficient retrieval we make three contributions: (i) we sparsify the descriptor points by locating discriminative regions within each image, thus reducing the computational expense of shape matching; (ii) we extend to enable matching despite changes in scale; and (iii) we show that vector quantizing the descriptor does not inhibit performance, thus providing the basis of a large-scale shape-based retrieval system using a bag-of-visual-words approach. Performance is demonstrated on the challenging ETHZ deformable shape dataset and a full episode from the television series Lost, and is shown to be superior to appearance-based approaches for matching non-rigid shape classes.