el 21(22): e5

Research Article

Six-layer Optimized Convolutional Neural Network for Lip Language Identification

Download286 downloads
  • @ARTICLE{10.4108/eai.20-8-2021.170751,
        author={Yifei Qiao and Hongli Chen and Xi Huang and Juan Lei and Xiangyu Cheng and Huibao Huang and Jinghan Wu and Xianwei Jiang},
        title={Six-layer Optimized Convolutional Neural Network for Lip Language Identification},
        journal={EAI Endorsed Transactions on e-Learning},
        keywords={lip language identification, convolutional neural network, Batch Normalization, dropout},
  • Yifei Qiao
    Hongli Chen
    Xi Huang
    Juan Lei
    Xiangyu Cheng
    Huibao Huang
    Jinghan Wu
    Xianwei Jiang
    Year: 2021
    Six-layer Optimized Convolutional Neural Network for Lip Language Identification
    DOI: 10.4108/eai.20-8-2021.170751
Yifei Qiao1, Hongli Chen1, Xi Huang1, Juan Lei1, Xiangyu Cheng1, Huibao Huang1, Jinghan Wu1, Xianwei Jiang1,*
  • 1: School of Mathematics and Information Science, Nanjing Normal University of Special Education, Nanjing 210038, China
*Contact email: jxw@njts.edu.cn


INTRODUCTION: Lip language is one of the most important communication methods in social life for people with hearing impairment and impaired expression ability. This communication method relies on visual recognition to understand the meaning expressed in communication.

OBJECTIVES: In order to improve the accuracy of this natural language recognition, we propose six-layer optimized convolutional neural network for lip recognition.

METHODS: The calculation method of the convolutional layer in the CNN model is used, and two pooling methods are compared: the maximum pooling operation and the average pooling operation to analyse the most important feature data in the picture. In order to reduce the simulation in the model training process, the closing rate has been optimized by introducing Dropout technology.

RESULTS: It shows that the recognition accuracy rate based on the six-layer convolutional neural network can reach 85.74% on average. This method can effectively recognize lip language.

CONCLUSION: We propose a six-layer optimized convolutional neural network method for lip language recognition, and the identification of lip language features of this method is better than 3D+ DenseNet +1 × 1 Conv +resBi-LSTM, 3D+CNN, ConvNet+2 -256-LSTM+VGG-16 three advanced methods.