English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Video outpainting aims to adequately complete missing areas at the edges ofvideo frames. Compared to image outpainting, it presents an additionalchallenge as the model should maintain the temporal consistency of the filledarea. In this paper, we introduce a masked 3D diffusion model for videooutpainting. We use the technique of mask modeling to train the 3D diffusionmodel. This allows us to use multiple guide frames to connect the results ofmultiple video clip inferences, thus ensuring temporal consistency and reducingjitter between adjacent frames. Meanwhile, we extract the global frames of thevideo as prompts and guide the model to obtain information other than thecurrent video clip using cross-attention. We also introduce a hybridcoarse-to-fine inference pipeline to alleviate the artifact accumulationproblem. The existing coarse-to-fine pipeline only uses the infilling strategy,which brings degradation because the time interval of the sparse frames is toolarge. Our pipeline benefits from bidirectional learning of the mask modelingand thus can employ a hybrid strategy of infilling and interpolation whengenerating sparse frames. Experiments show that our method achievesstate-of-the-art results in video outpainting tasks. More results and codes areprovided at our https://fanfanda.github.io/M3DDM/.

Hierarchical Masked 3D Diffusion Model for Video Outpainting