English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Forced alignment refers to a technology that time-aligns a giventranscription with a corresponding speech. However, as the forced alignmenttechnologies have developed using speech audio, they might fail in alignmentwhen the input speech audio is noise-corrupted or is not accessible. We focuson that there is another component that the speech can be inferred from, thespeech video (i.e., talking face video). Since the drawbacks of audio-basedforced alignment can be complemented using the visual information when theaudio signal is under poor condition, we try to develop a novel video-basedforced alignment method. However, different from audio forced alignment, it ischallenging to develop a reliable visual forced alignment technology for thefollowing two reasons: 1) Visual Speech Recognition (VSR) has a much lowerperformance compared to audio-based Automatic Speech Recognition (ASR), and 2)the translation from text to video is not reliable, so the method typicallyused for building audio forced alignment cannot be utilized in developingvisual forced alignment. In order to alleviate these challenges, in this paper,we propose a new method that is appropriate for visual forced alignment, namelyDeep Visual Forced Alignment (DVFA). The proposed DVFA can align the inputtranscription (i.e., sentence) with the talking face video without accessingthe speech audio. Moreover, by augmenting the alignment task with anomaly casedetection, DVFA can detect mismatches between the input transcription and theinput video while performing the alignment. Therefore, we can robustly alignthe text with the talking face video even if there exist error words in thetext. Through extensive experiments, we show the effectiveness of the proposedDVFA not only in the alignment task but also in interpreting the outputs of VSRmodels.

Deep Visual Forced Alignment: Learning to Align Transcription with Talking Face Video