English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

This paper identifies two kinds of redundancy in the current VideoQAparadigm. Specifically, the current video encoders tend to holistically embedall video clues at different granularities in a hierarchical manner, whichinevitably introduces \textit{neighboring-frame redundancy} that can overwhelmdetailed visual clues at the object level. Subsequently, prevailingvision-language fusion designs introduce the \textit{cross-modal redundancy} byexhaustively fusing all visual elements with question tokens without explicitlydifferentiating their pairwise vision-language interactions, thus making apernicious impact on the answering.  To this end, we propose a novel transformer-based architecture, that aims tomodel VideoQA in a redundancy-aware manner. To address the neighboring-frameredundancy, we introduce a video encoder structure that emphasizes theobject-level change in neighboring frames, while adopting an out-of-neighboringmessage-passing scheme that imposes attention only on distant frames. As forthe cross-modal redundancy, we equip our fusion module with a novel adaptivesampling, which explicitly differentiates the vision-language interactions byidentifying a small subset of visual elements that exclusively support theanswer. Upon these advancements, we find this\underline{R}edundancy-\underline{a}ware trans\underline{former} (RaFormer) canachieve state-of-the-art results on multiple VideoQA benchmarks.

Redundancy-aware Transformer for Video Question Answering