English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce a multi-modal diffusion model tailored for the bi-directionalconditional generation of video and audio. Recognizing the importance ofaccurate alignment between video and audio events in multi-modal generationtasks, we propose a joint contrastive training loss to enhance thesynchronization between visual and auditory occurrences. Our researchmethodology involves conducting comprehensive experiments on multiple datasetsto thoroughly evaluate the efficacy of our proposed model. The assessment ofgeneration quality and alignment performance is carried out from variousangles, encompassing both objective and subjective metrics. Our findingsdemonstrate that the proposed model outperforms the baseline, substantiatingits effectiveness and efficiency. Notably, the incorporation of the contrastiveloss results in improvements in audio-visual alignment, particularly in thehigh-correlation video-to-audio generation task. These results indicate thepotential of our proposed model as a robust solution for improving the qualityand alignment of multi-modal generation, thereby contributing to theadvancement of video and audio conditional generation systems.

CMMD: Contrastive Multi-Modal Diffusion for Video-Audio Conditional Modeling