Sketch-to-photo matching: a feature-based approach

This paper presents a local feature-based method for matching facial sketch images to face photographs, which is the first known feature-based method for performing such matching. Starting with a training set of sketch to photo correspondences (i.e. a set of sketch and photo images of the same subjects), we demonstrate the ability to match sketches to photos: (1) directly using SIFT feature descriptors, (2) in a "common representation" that measures the similarity between a sketch and photo by their distance from the training set of sketch/photo pairs, and (3) by fusing the previous two methods. For both matching methods, the first step is to sample SIFT feature descriptors uniformly across all the sketch and photo images. In direct matching, we simply measure the distance of the SIFT descriptors between sketches and photos. In common representation matching, the distance between the descriptor vectors of the probe sketches and gallery photos at each local sample point is measured. This results in a vector of distances across the sketch or photo image to each member of the training basis. Further recognition improvements are shown by score level fusion of the two sketch matchers. Compared with published sketch to photo matching algorithms, experimental results demonstrate improved matching performances using the presented feature-based methods.