English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Most existing video diffusion models (VDMs) are limited to mere textconditions. Thereby, they are usually lacking in control over visual appearanceand geometry structure of the generated videos. This work presents Moonshot, anew video generation model that conditions simultaneously on multimodal inputsof image and text. The model builts upon a core module, called multimodal videoblock (MVB), which consists of conventional spatialtemporal layers forrepresenting video features, and a decoupled cross-attention layer to addressimage and text inputs for appearance conditioning. In addition, we carefullydesign the model architecture such that it can optionally integrate withpre-trained image ControlNet modules for geometry visual conditions, withoutneeding of extra training overhead as opposed to prior methods. Experimentsshow that with versatile multimodal conditioning mechanisms, Moonshotdemonstrates significant improvement on visual quality and temporal consistencycompared to existing models. In addition, the model can be easily repurposedfor a variety of generative applications, such as personalized videogeneration, image animation and video editing, unveiling its potential to serveas a fundamental architecture for controllable video generation. Models will bemade public on https://github.com/salesforce/LAVIS.

Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions