English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Audio-visual automatic speech recognition (AV-ASR) introduces the videomodality into the speech recognition process, often by relying on informationconveyed by the motion of the speaker's mouth. The use of the video signalrequires extracting visual features, which are then combined with the acousticfeatures to build an AV-ASR system [1]. This is traditionally done with someform of 3D convolutional network (e.g. VGG) as widely used in the computervision community. Recently, image transformers [2] have been introduced toextract visual features useful for image classification tasks. In this work, wepropose to replace the 3D convolutional visual front-end with a videotransformer front-end. We train our systems on a large-scale dataset composedof YouTube videos and evaluate performance on the publicly available LRS3-TEDset, as well as on a large set of YouTube videos. On a lip-reading task, thetransformer-based front-end shows superior performance compared to a strongconvolutional baseline. On an AV-ASR task, the transformer front-end performsas well as (or better than) the convolutional baseline. Fine-tuning our modelon the LRS3-TED training set matches previous state of the art. Thus, weexperimentally show the viability of the convolution-free model for AV-ASR.

Audio-Visual Speech Recognition is Worth 32$\times$32$\times$8 Voxels