English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multi-modal based speech separation has exhibited a specific advantage onisolating the target character in multi-talker noisy environments.Unfortunately, most of current separation strategies prefer a straightforwardfusion based on feature learning of each single modality, which is far fromsufficient consideration of inter-relationships between modalites. Inspired bylearning joint feature representations from audio and visual streams withattention mechanism, in this study, a novel cross-modal fusion strategy isproposed to benefit the whole framework with semantic correlations betweendifferent modalities. To further improve audio-visual speech separation, thedense optical flow of lip motion is incorporated to strengthen the robustnessof visual representation. The evaluation of the proposed work is performed ontwo public audio-visual speech separation benchmark datasets. The overallimprovement of the performance has demonstrated that the additional motionnetwork effectively enhances the visual representation of the combined lipimages and audio signal, as well as outperforming the baseline in terms of allmetrics with the proposed cross-modal fusion.

Audio-visual speech separation based on joint feature representation with cross-modal attention