English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

While automatic speech recognition (ASR) systems degrade significantly innoisy environments, audio-visual speech recognition (AVSR) systems aim tocomplement the audio stream with noise-invariant visual cues and improve thesystem's robustness. However, current studies mainly focus on fusing thewell-learned modality features, like the output of modality-specific encoders,without considering the contextual relationship during the modality featurelearning. In this study, we propose a multi-layer cross-attention fusion basedAVSR (MLCA-AVSR) approach that promotes representation learning of eachmodality by fusing them at different levels of audio/visual encoders.Experimental results on the MISP2022-AVSR Challenge dataset show the efficacyof our proposed system, achieving a concatenated minimum permutation charactererror rate (cpCER) of 30.57% on the Eval set and yielding up to 3.17% relativeimprovement compared with our previous system which ranked the second place inthe challenge. Following the fusion of multiple systems, our proposed approachsurpasses the first-place system, establishing a new SOTA cpCER of 29.13% onthis dataset.

MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition