English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Visual Speech Recognition (VSR) is a task to predict a sentence or word fromlip movements. Some works have been recently presented which use audio signalsto supplement visual information. However, existing methods utilize onlylimited information such as phoneme-level features and soft labels of AutomaticSpeech Recognition (ASR) networks. In this paper, we present a Multi-TemporalLip-Audio Memory (MTLAM) that makes the best use of audio signals to complementinsufficient information of lip movements. The proposed method is mainlycomposed of two parts: 1) MTLAM saves multi-temporal audio features producedfrom short- and long-term audio signals, and the MTLAM memorizes avisual-to-audio mapping to load stored multi-temporal audio features fromvisual features at the inference phase. 2) We design an audio temporal model toproduce multi-temporal audio features capturing the context of neighboringwords. In addition, to construct effective visual-to-audio mapping, the audiotemporal models can generate audio features time-aligned with visual features.Through extensive experiments, we validate the effectiveness of the MTLAMachieving state-of-the-art performances on two public VSR datasets.

Multi-Temporal Lip-Audio Memory for Visual Speech Recognition