English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present Compound Conditioned ControlNet, C3Net, a novel generative neuralarchitecture taking conditions from multiple modalities and synthesizingmultimodal contents simultaneously (e.g., image, text, audio). C3Net adapts theControlNet architecture to jointly train and make inferences on aproduction-ready diffusion model and its trainable copies. Specifically, C3Netfirst aligns the conditions from multi-modalities to the same semantic latentspace using modality-specific encoders based on contrastive training. Then, itgenerates multimodal outputs based on the aligned latent space, whose semanticinformation is combined using a ControlNet-like architecture called ControlC3-UNet. Correspondingly, with this system design, our model offers an improvedsolution for joint-modality generation through learning and explainingmultimodal conditions instead of simply taking linear interpolations on thelatent space. Meanwhile, as we align conditions to a unified latent space,C3Net only requires one trainable Control C3-UNet to work on multimodalsemantic information. Furthermore, our model employs unimodal pretraining onthe condition alignment stage, outperforming the non-pretrained alignment evenon relatively scarce training data and thus demonstrating high-quality compoundcondition generation. We contribute the first high-quality tri-modal validationset to validate quantitatively that C3Net outperforms or is on par with firstand contemporary state-of-the-art multimodal generation. Our codes andtri-modal dataset will be released.

C3Net: Compound Conditioned ControlNet for Multimodal Content Generation