Open AccessSequential particle filter with covariance features classified with artificial neural nets for continuous Indian sign language recognition
Author(s) -
Pramod Kumar,
P. Jayarama Reddy,
P. Srinivasa Rao
Publication year - 2017
Publication title -
international journal of engineering and technology
Language(s) - English
Resource type - Journals
ISSN - 2227-524X
DOI - 10.14419/ijet.v7i1.1.10163
Subject(s) - computer science , artificial intelligence , sign language , pattern recognition (psychology) , classifier (uml) , backpropagation , feature (linguistics) , speech recognition , covariance matrix , artificial neural network , covariance , particle filter , algorithm , mathematics , kalman filter , philosophy , linguistics , statistics
Machine translation of sign language is a complex and challenging problem in computer vision research. In this work, we propose to handle issues such as hands tracking, feature representation and classification for efficient interpretation of sign language from isolated sign videos. Hands tracking is attempted in a sequential format with one hand after the other by nullifying the effects of head movement using serial particle filter. The estimated hand positions in the video sequence are used to extract the hand portions to create a feature covariance matrix. This matrix is a compact representation of the hand features representing a sign. Adaptability of the feature covariance matrix is explored in developing relationships with new signs without creating a new feature matrix for individual signs. The extracted features are then applied to a neural network classifier which is trained with error backpropagation algorithm. Multiple experiments were conducted on a 181 class signs with 50 sentence formations with 5 different signers. Experimental results show the proposed sequential hand tracking is closer to ground truth. The proposed covariance features resulted in a classification accuracy of 89.34% with the neural network classifier.