English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Existing methods for video question answering (VideoQA) often suffer fromspurious correlations between different modalities, leading to a failure inidentifying the dominant visual evidence and the intended question. Moreover,these methods function as black boxes, making it difficult to interpret thevisual scene during the QA process. In this paper, to discover critical videosegments and frames that serve as the visual causal scene for generatingreliable answers, we present a causal analysis of VideoQA and propose aframework for cross-modal causal relational reasoning, named Visual CausalScene Refinement (VCSR). Particularly, a set of causal front-door interventionoperations is introduced to explicitly find the visual causal scenes at bothsegment and frame levels. Our VCSR involves two essential modules: i) theQuestion-Guided Refiner (QGR) module, which refines consecutive video framesguided by the question semantics to obtain more representative segment featuresfor causal front-door intervention; ii) the Causal Scene Separator (CSS)module, which discovers a collection of visual causal and non-causal scenesbased on the visual-linguistic causal relevance and estimates the causal effectof the scene-separating intervention in a contrastive learning manner.Extensive experiments on the NExT-QA, Causal-VidQA, and MSRVTT-QA datasetsdemonstrate the superiority of our VCSR in discovering visual causal scene andachieving robust video question answering. The code is available athttps://github.com/YangLiu9208/VCSR.

Visual Causal Scene Refinement for Video Question Answering