Answers are grounded in this paper and cite their sources.
We examine the effect of depth information captured by Microsoft Kinect on the task of visual speech recognition. We propose depth-based active appearance model (AAM) features and show improved results over discrete cosine transform (DCT). The visual and depth features are evaluated on a database of 54 speakers each uttering 50 isolated words. In order to exploit the speech dynamics, the features are enhanced by a simplified one-stage variant of hierarchical linear discriminant analysis (Hi-LDA). In the experiments, we consider feature fusion via combined video-depth active appearance model as a form of early integration, and compare it to traditional multi-stream hidden Markov Model as a form of decision fusion. We also perform experiments on audio-visual recognition in noisy environments and show improved results of incorporating depth information over both traditional audio-video fusion and utilization of speech enhancement algorithms.
No takes yet. Share an insight, caveat, or question.
Karel Paleček (2015) studied this question.
Synapse has enriched 3 closely related papers on similar clinical questions. Consider them for comparative context: