English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Sign Language (SL), as the mother tongue of the deaf community, is a specialvisual language that most hearing people cannot understand. In recent years,neural Sign Language Translation (SLT), as a possible way for bridgingcommunication gap between the deaf and the hearing people, has attractedwidespread academic attention. We found that the current mainstream end-to-endneural SLT models, which tries to learning language knowledge in a weaklysupervised manner, could not mine enough semantic information under thecondition of low data resources. Therefore, we propose to introduce additionalword-level semantic knowledge of sign language linguistics to assist inimproving current end-to-end neural SLT models. Concretely, we propose a novelneural SLT model with multi-modal feature fusion based on the dynamic graph, inwhich the cross-modal information, i.e. text and video, is first assembled as adynamic graph according to their correlation, and then the graph is processedby a multi-modal graph encoder to generate the multi-modal embeddings forfurther usage in the subsequent neural translation models. To the best of ourknowledge, we are the first to introduce graph neural networks, for fusingmulti-modal information, into neural sign language translation models.Moreover, we conducted experiments on a publicly available popular SLT datasetRWTH-PHOENIX-Weather-2014T. and the quantitative experiments show that ourmethod can improve the model.

Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation