English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformer, as a strong and flexible architecture for modelling long-rangerelations, has been widely explored in vision tasks. However, when used invideo inpainting that requires fine-grained representation, existed methodstill suffers from yielding blurry edges in detail due to the hard patchsplitting. Here we aim to tackle this problem by proposing FuseFormer, aTransformer model designed for video inpainting via fine-grained feature fusionbased on novel Soft Split and Soft Composition operations. The soft splitdivides feature map into many patches with given overlapping interval. On thecontrary, the soft composition operates by stitching different patches into awhole feature map where pixels in overlapping regions are summed up. These twomodules are first used in tokenization before Transformer layers andde-tokenization after Transformer layers, for effective mapping between tokensand features. Therefore, sub-patch level information interaction is enabled formore effective feature propagation between neighboring patches, resulting insynthesizing vivid content for hole regions in videos. Moreover, in FuseFormer,we elaborately insert the soft composition and soft split into the feed-forwardnetwork, enabling the 1D linear layers to have the capability of modelling 2Dstructure. And, the sub-patch level feature fusion ability is further enhanced.In both quantitative and qualitative evaluations, our proposed FuseFormersurpasses state-of-the-art methods. We also conduct detailed analysis toexamine its superiority.

FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting