English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multimodal representation learning poses significant challenges in capturinginformative and distinct features from multiple modalities. Existing methodsoften struggle to exploit the unique characteristics of each modality due tounified multimodal annotations. In this study, we propose Self-MI in theself-supervised learning fashion, which also leverage Contrastive PredictiveCoding (CPC) as an auxiliary technique to maximize the Mutual Information (MI)between unimodal input pairs and the multimodal fusion result with unimodalinputs. Moreover, we design a label generation module, $ULG_{MI}$ for short,that enables us to create meaningful and informative labels for each modalityin a self-supervised manner. By maximizing the Mutual Information, we encouragebetter alignment between the multimodal fusion and the individual modalities,facilitating improved multimodal fusion. Extensive experiments on threebenchmark datasets including CMU-MOSI, CMU-MOSEI, and SIMS, demonstrate theeffectiveness of Self-MI in enhancing the multimodal fusion task.

Self-MI: Efficient Multimodal Fusion via Self-Supervised Multi-Task Learning with Auxiliary Mutual Information Maximization