English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Sign language recognition (SLR) is a weakly supervised task that annotatessign videos as textual glosses. Recent studies show that insufficient trainingcaused by the lack of large-scale available sign datasets becomes the mainbottleneck for SLR. Most SLR works thereby adopt pretrained visual modules anddevelop two mainstream solutions. The multi-stream architectures extendmulti-cue visual features, yielding the current SOTA performances but requiringcomplex designs and might introduce potential noise. Alternatively, theadvanced single-cue SLR frameworks using explicit cross-modal alignment betweenvisual and textual modalities are simple and effective, potentially competitivewith the multi-cue framework. In this work, we propose a novel contrastivevisual-textual transformation for SLR, CVT-SLR, to fully explore the pretrainedknowledge of both the visual and language modalities. Based on the single-cuecross-modal alignment framework, we propose a variational autoencoder (VAE) forpretrained contextual knowledge while introducing the complete pretrainedlanguage module. The VAE implicitly aligns visual and textual modalities whilebenefiting from pretrained contextual knowledge as the traditional contextualmodule. Meanwhile, a contrastive cross-modal alignment algorithm is designed toexplicitly enhance the consistency constraints. Extensive experiments on publicdatasets (PHOENIX-2014 and PHOENIX-2014T) demonstrate that our proposed CVT-SLRconsistently outperforms existing single-cue methods and even outperforms SOTAmulti-cue methods.

CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment