Open AccessHow important is motion in sign language translation?
Author(s) -
Rodriguez Jefferson,
Martínez Fabio
Publication year - 2021
Publication title -
iet computer vision
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.38
H-Index - 37
eISSN - 1751-9640
pISSN - 1751-9632
DOI - 10.1049/cvi2.12037
Subject(s) - computer science , sign language , sign (mathematics) , motion (physics) , set (abstract data type) , machine translation , artificial intelligence , translation (biology) , representation (politics) , natural language processing , variance (accounting) , architecture , test set , speech recognition , linguistics , mathematics , art , biochemistry , chemistry , accounting , politics , political science , law , business , visual arts , gene , programming language , mathematical analysis , philosophy , messenger rna
More than 70 million people use at least one sign language (SL) as their main channel of communication. Nevertheless, the absence of effective mechanisms to translate massive information among sign, written and spoken languages is the main cause of a negligible inclusion of deaf people into society. Therefore, SL automatic recognition systems have widely proposed to support the characterisation of the sign structure. Today, natural and continuous SL recognition is an open research problem due to multiple spatio‐temporal shape variations, challenging visual sign characterisation, as well as the non‐linear correlation among signs to express a message. A compact sign is introduced to text architecture that explores motion as an alternative to support sign translation. Such characterisation results are robust to appearance variance with relative support to geometrical variations. The proposed representation focuses on the main spatio‐temporal regions to each corresponding word. The proposed architecture was evaluated in a built SL data set (LSCDv1) dedicated to motion study and also in the state‐of‐the‐art RWTH‐Phoenix. From the LSCDv1 data set, the best configuration reports a BLEU‐4 score of 63.04 in a testing set. Regarding RWTH‐Phoenix, the proposed strategy achieved a BLEU‐4 score in a test of 4.56, improving the results under similar reduced conditions.