English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper describes an audio-visual speech enhancement (AV-SE) method thatestimates from noisy input audio a mixture of the speech of the speakerappearing in an input video (on-screen target speech) and of a selected speakernot appearing in the video (off-screen target speech). Although conventionalAV-SE methods have suppressed all off-screen sounds, it is necessary to listento a specific pre-known speaker's speech (e.g., family member's voice andannouncements in stations) in future applications of AV-SE (e.g., hearingaids), even when users' sight does not capture the speaker. To overcome thislimitation, we extract a visual clue for the on-screen target speech from theinput video and a voiceprint clue for the off-screen one from a pre-recordedspeech of the speaker. Two clues from different domains are integrated as anaudio-visual clue, and the proposed model directly estimates the targetmixture. To improve the estimation accuracy, we introduce a temporal attentionmechanism for the voiceprint clue and propose a training strategy called themuting strategy. Experimental results show that our method outperforms abaseline method that uses the state-of-the-art AV-SE and speaker extractionmethods individually in terms of estimation accuracy and computationalefficiency.

Audio-Visual Speech Enhancement With Selective Off-Screen Speech Extraction