Research Article
A Robust Visual Feature Extraction based BTSM-LDA for Audio-Visual Speech Recognition
@INPROCEEDINGS{10.1109/CHINACOM.2007.4469472, author={Guoyun Lv and Rongchun Zhao and Dongmei Jiang and Yan Li and H. Sahli}, title={A Robust Visual Feature Extraction based BTSM-LDA for Audio-Visual Speech Recognition}, proceedings={2nd International ICST Conference on Communications and Networking in China}, publisher={IEEE}, proceedings_a={CHINACOM}, year={2008}, month={3}, keywords={Bayesian Tangent Shape Model Dynamic Bayesian Networks audio-visual speech recognition}, doi={10.1109/CHINACOM.2007.4469472} }- Guoyun Lv
Rongchun Zhao
Dongmei Jiang
Yan Li
H. Sahli
Year: 2008
A Robust Visual Feature Extraction based BTSM-LDA for Audio-Visual Speech Recognition
CHINACOM
IEEE
DOI: 10.1109/CHINACOM.2007.4469472
Abstract
The asynchrony for speech and lip movement is key problem of audio-visual speech recognition (AVSR) system. A Multi-Stream Asynchrony Dynamic Bayesian Network (MSADBN) model is proposed for audio-visual speech recognition. Comparing with Multi-Stream HMM (MSHMM), MS-ADBN model describes the asynchrony of audio stream and visual stream to the word level. Simultaneously, based on profile of lip implemented by using Bayesian Tangent Shape Model (BTSM), Linear Discrimination Analysis (LDA) is used for visual feature extraction which describes the dynamic feature of lip and removes the redundancy of lip geometrical feature. The experiments results on continuous digit audio-visual database show that lip dynamic feature based on BTSM and LDA is more stable and robust than direct lip geometrical feature. In the noisy environments with signal to noise ratios ranging from 0dB to 30dB, comparing with MSHMM, MS-ADBN model with MFCC and LDA visual features has an average improvement of 4.92% in speech recognition rate.
