English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multimodal learning (MML) aims to jointly exploit the common priors ofdifferent modalities to compensate for their inherent limitations. However,existing MML methods often optimize a uniform objective for differentmodalities, leading to the notorious "modality imbalance" problem andcounterproductive MML performance. To address the problem, some existingmethods modulate the learning pace based on the fused modality, which isdominated by the better modality and eventually results in a limitedimprovement on the worse modal. To better exploit the features of multimodal,we propose Prototypical Modality Rebalance (PMR) to perform stimulation on theparticular slow-learning modality without interference from other modalities.Specifically, we introduce the prototypes that represent general features foreach class, to build the non-parametric classifiers for uni-modal performanceevaluation. Then, we try to accelerate the slow-learning modality by enhancingits clustering toward prototypes. Furthermore, to alleviate the suppressionfrom the dominant modality, we introduce a prototype-based entropyregularization term during the early training stage to prevent prematureconvergence. Besides, our method only relies on the representations of eachmodality and without restrictions from model structures and fusion methods,making it with great application potential for various scenarios.

PMR: Prototypical Modal Rebalance for Multimodal Learning