Open AccessDevelopment of a Verbal Robot Hand Gesture Recognition System
Author(s) -
Chingis Kenshimov,
Talgat Sundetov,
Murat Kunelbayev,
Zhazira Amirgaliyeva,
Didar Yedilkhan,
Omirlan Auelbekov
Publication year - 2021
Publication title -
wseas transactions on systems and control/wseas transactions on systems and control
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.174
H-Index - 16
eISSN - 2224-2856
pISSN - 1991-8763
DOI - 10.37394/23203.2021.16.53
Subject(s) - gesture , computer science , gesture recognition , robot , artificial intelligence , python (programming language) , sign language , speech recognition , computer vision , programming language , linguistics , philosophy
This article analyzes the most famous sign languages, the correlation of sign languages, and also considers the development of a verbal robot hand gesture recognition system in relation to the Kazakh language. The proposed system contains a touch sensor, in which the contact of the electrical property of the user's skin is measured, which provides more accurate information for simulating and indicating the gestures of the robot hand. Within the framework of the system, the speed and accuracy of recognition of each gesture of the verbal robot are calculated. The average recognition accuracy was over 98%. The detection time was 3ms on a 1.9 GHz Jetson Nano processor, which is enough to create a robot showing natural language gestures. A complete fingerprint of the Kazakh sign language for a verbal robot is also proposed. To improve the quality of gesture recognition, a machine learning method was used. The operability of the developed technique for recognizing gestures by a verbal robot was tested, and on the basis of computational experiments, the effectiveness of algorithms and software for responding to a verbal robot to a voice command was evaluated based on automatic recognition of a multilingual human voice. Thus, we can assume that the authors have proposed an intelligent verbal complex implemented in Python with the CMUSphinx communication module and the PyOpenGL graphical command execution simulator. Robot manipulation module based on 3D modeling from ABB.