English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Video moment localization aims to retrieve the target segment of an untrimmedvideo according to the natural language query. Weakly supervised methods gainsattention recently, as the precise temporal location of the target segment isnot always available. However, one of the greatest challenges encountered bythe weakly supervised method is implied in the mismatch between the video andlanguage induced by the coarse temporal annotations. To refine thevision-language alignment, recent works contrast the cross-modalitysimilarities driven by reconstructing masked queries between positive andnegative video proposals. However, the reconstruction may be influenced by thelatent spurious correlation between the unmasked and the masked parts, whichdistorts the restoring process and further degrades the efficacy of contrastivelearning since the masked words are not completely reconstructed from thecross-modality knowledge. In this paper, we discover and mitigate this spuriouscorrelation through a novel proposed counterfactual cross-modality reasoningmethod. Specifically, we first formulate query reconstruction as an aggregatedcausal effect of cross-modality and query knowledge. Then by introducingcounterfactual cross-modality knowledge into this aggregation, the spuriousimpact of the unmasked part contributing to the reconstruction is explicitlymodeled. Finally, by suppressing the unimodal effect of masked query, we canrectify the reconstructions of video proposals to perform reasonablecontrastive learning. Extensive experimental evaluations demonstrate theeffectiveness of our proposed method. The code is available at\href{https://github.com/sLdZ0306/CCR}{https://github.com/sLdZ0306/CCR}.

Counterfactual Cross-modality Reasoning for Weakly Supervised Video Moment Localization