Spatiotemporal analysis of face profiles: detection, segmentation, and registration

We use a two-image approach to construct a 3D human facial model for multimedia applications. The images used are those of faces at direct frontal and side views. The selection of the side view from a sequence of facial images is automatically done by applying a spatiotemporal approach to face profile analysis. The extracted side profile is then segmented based on knowledge of known local facial structures. Temporal consistency in the structure of the face under small deviations from the profile image is exploited to improve robustness of the segmentation in the presence of noise and image artifacts. Once the face profile is properly segmented, nine perceptually significant landmarks (fiducial points) on the face are registered based on a local maximum curvature computation. The methods developed are invariant to geometric transformation and are more robust than earlier approaches which rely only on the spatial structure of the face.