English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformers have been widely used for video processing owing to themulti-head self attention (MHSA) mechanism. However, the MHSA mechanismencounters an intrinsic difficulty for video inpainting, since the featuresassociated with the corrupted regions are degraded and incur inaccurate selfattention. This problem, termed query degradation, may be mitigated by firstcompleting optical flows and then using the flows to guide the self attention,which was verified in our previous work - flow-guided transformer (FGT). Wefurther exploit the flow guidance and propose FGT++ to pursue more effectiveand efficient video inpainting. First, we design a lightweight flow completionnetwork by using local aggregation and edge loss. Second, to address the querydegradation, we propose a flow guidance feature integration module, which usesthe motion discrepancy to enhance the features, together with a flow-guidedfeature propagation module that warps the features according to the flows.Third, we decouple the transformer along the temporal and spatial dimensions,where flows are used to select the tokens through a temporally deformable MHSAmechanism, and global tokens are combined with the inner-window local tokensthrough a dual perspective MHSA mechanism. FGT++ is experimentally evaluated tobe outperforming the existing video inpainting networks qualitatively andquantitatively.

Exploiting Optical Flow Guidance for Transformer-Based Video Inpainting