English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we devise a mechanism for the addition of multi-modalinformation with an existing pipeline for continuous sign language recognitionand translation. In our procedure, we have incorporated optical flowinformation with RGB images to enrich the features with movement-relatedinformation. This work studies the feasibility of such modality inclusion usinga cross-modal encoder. The plugin we have used is very lightweight and doesn'tneed to include a separate feature extractor for the new modality in anend-to-end manner. We have applied the changes in both sign languagerecognition and translation, improving the result in each case. We haveevaluated the performance on the RWTH-PHOENIX-2014 dataset for sign languagerecognition and the RWTH-PHOENIX-2014T dataset for translation. On therecognition task, our approach reduced the WER by 0.9, and on the translationtask, our approach increased most of the BLEU scores by ~0.6 on the test set.

Attention-Driven Multi-Modal Fusion: Enhancing Sign Language Recognition and Translation