English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Text-to-video editing aims to edit the visual appearance of a source videoconditional on textual prompts. A major challenge in this task is to ensurethat all frames in the edited video are visually consistent. Most recent worksapply advanced text-to-image diffusion models to this task by inflating 2Dspatial attention in the U-Net into spatio-temporal attention. Althoughtemporal context can be added through spatio-temporal attention, it mayintroduce some irrelevant information for each patch and therefore causeinconsistency in the edited video. In this paper, for the first time, weintroduce optical flow into the attention module in the diffusion model's U-Netto address the inconsistency issue for text-to-video editing. Our method,FLATTEN, enforces the patches on the same flow path across different frames toattend to each other in the attention module, thus improving the visualconsistency in the edited videos. Additionally, our method is training-free andcan be seamlessly integrated into any diffusion-based text-to-video editingmethods and improve their visual consistency. Experiment results on existingtext-to-video editing benchmarks show that our proposed method achieves the newstate-of-the-art performance. In particular, our method excels in maintainingthe visual consistency in the edited videos.

FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editing