Open AccessWord recognition through mapping of lip movements from speech utterance using audiovisual fusion and MLP
Author(s) -
G N Srikanth,
M. K. Venkatesha
Publication year - 2022
Publication title -
international journal of health sciences (ijhs) (en línea)
Language(s) - English
Resource type - Journals
eISSN - 2550-6978
pISSN - 2550-696X
DOI - 10.53730/ijhs.v6ns2.6078
Subject(s) - computer science , speech recognition , utterance , artificial intelligence , mel frequency cepstrum , classifier (uml) , feature extraction , word recognition , word (group theory) , perceptron , support vector machine , pattern recognition (psychology) , feature (linguistics) , audio mining , speech processing , artificial neural network , voice activity detection , reading (process) , linguistics , philosophy , political science , law
Speech has information more than text, but under noisy environment speech sufferance from disadvantage of not properly decoded by humans and same is true with machines. speech being bimodal along with audio features if we augment visual features specifically related to lip movements. the degree of speech recognition can be improved. The objective of this work is to use audio and visual features to aid word recognition. In this work we extracted MFCC features for audio and Geometrical features of lip movements together is used in machine learning algorithm to predict the word utterances. Videos related to word utterances are extracted from TIMID database. With the statistical information related to audio and corresponding visual features from lip movements is extracted to form input feature vector to machine learning algorithm (Multi-layer perceptron). The experimental results show that using MLP we have obtained a word recognition accuracy of 91% and using KNN Classifier the accuracy attained is 61%. The results presented here have important implications for applications in HMI communication and helps hearing impaired.