A vision system performing lip shape analysis for speech pathology research

We have built a machine vision system to perform lip shape analysis and recognition and to generate 3-D lip models for speech pathology research. The system employs two color cameras that are synchronized to capture image video pairs simultaneously. A novel color segmentation technique that converts the RGB color components to linear color space has been developed to extract lip shape contours. Lip shape of the viseme, the smallest visibly distinguishable unit of speech, is used to identify its corresponding image frame in the recorded image sequences. It is also used to reconstruct the 3-D lip shape model as a quantitative description of the visible aspects of spoken language for speech pathology research.