English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Audio-visual information fusion enables a performance improvement in speechrecognition performed in complex acoustic scenarios, e.g., noisy environments.It is required to explore an effective audio-visual fusion strategy foraudiovisual alignment and modality reliability. Different from the previousend-to-end approaches where the audio-visual fusion is performed after encodingeach modality, in this paper we propose to integrate an attentive fusion blockinto the encoding process. It is shown that the proposed audio-visual fusionmethod in the encoder module can enrich audio-visual representations, as therelevance between the two modalities is leveraged. In line with thetransformer-based architecture, we implement the embedded fusion block using amulti-head attention based audiovisual fusion with one-way or two-wayinteractions. The proposed method can sufficiently combine the two streams andweaken the over-reliance on the audio modality. Experiments on the LRS3-TEDdataset demonstrate that the proposed method can increase the recognition rateby 0.55%, 4.51% and 4.61% on average under the clean, seen and unseen noiseconditions, respectively, compared to the state-of-the-art approach.

Attentive Fusion Enhanced Audio-Visual Encoding for Transformer Based Robust Speech Recognition