English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Video multimodal fusion aims to integrate multimodal signals in videos, suchas visual, audio and text, to make a complementary prediction with multiplemodalities contents. However, unlike other image-text multimodal tasks, videohas longer multimodal sequences with more redundancy and noise in both visualand audio modalities. Prior denoising methods like forget gate are coarse inthe granularity of noise filtering. They often suppress the redundant and noisyinformation at the risk of losing critical information. Therefore, we propose adenoising bottleneck fusion (DBF) model for fine-grained video multimodalfusion. On the one hand, we employ a bottleneck mechanism to filter out noiseand redundancy with a restrained receptive field. On the other hand, we use amutual information maximization module to regulate the filter-out module topreserve key information within different modalities. Our DBF model achievessignificant improvement over current state-of-the-art baselines on multiplebenchmarks covering multimodal sentiment analysis and multimodal summarizationtasks. It proves that our model can effectively capture salient features fromnoisy and redundant video, audio, and text inputs. The code for this paper ispublicly available at https://github.com/WSXRHFG/DBF.

Denoising Bottleneck with Mutual Information Maximization for Video Multimodal Fusion