Open AccessGesture identification in testing mode in the integrated sign language training system
Author(s) -
Mikhail G. Grif,
A. V. Lukoyanychev
Publication year - 2020
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1661/1/012037
Subject(s) - gesture , computer science , identification (biology) , mode (computer interface) , sign language , avatar , gesture recognition , similarity (geometry) , projection (relational algebra) , artificial intelligence , fingerprint (computing) , word (group theory) , sign (mathematics) , computer vision , speech recognition , sign system , pattern recognition (psychology) , natural language processing , human–computer interaction , algorithm , linguistics , image (mathematics) , mathematics , mathematical analysis , philosophy , botany , biology
The structure of an integral sign language teaching system based on a 3D computer character, which allows demonstrating fingerprints and words of a sign language using an avatar, is considered. The system provides the test mode when the user must demonstrate the word or fingerprint indicated by the system. The user feedback with the system is carried out using a standard webcam. The problems of gesture localization and identification in user testing mode are considered. The features of this mode are determined during the system operation. The algorithm for quick identification of human hand gestures is proposed. The method based on the signs, which are two-dimensional projections, is used for gesture identification. The projection recognition features are considered, the similarity metrics are analyzed, and three stages for match decision are identified. The experimental results show the stable operation of this algorithm in various lighting conditions and at the place where the gesture was shown.