English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Reasoning about causal and temporal event relations in videos is a newdestination of Video Question Answering (VideoQA).The major stumbling block toachieve this purpose is the semantic gap between language and video since theyare at different levels of abstraction. Existing efforts mainly focus ondesigning sophisticated architectures while utilizing frame- or object-levelvisual representations. In this paper, we reconsider the multi-modal alignmentproblem in VideoQA from feature and sample perspectives to achieve betterperformance. From the view of feature,we break down the video into trajectoriesand first leverage trajectory feature in VideoQA to enhance the alignmentbetween two modalities. Moreover, we adopt a heterogeneous graph architectureand design a hierarchical framework to align both trajectory-level andframe-level visual feature with language feature. In addition, we found thatVideoQA models are largely dependent on language priors and always neglectvisual-language interactions. Thus, two effective yet portable trainingaugmentation strategies are designed to strengthen the cross-modalcorrespondence ability of our model from the view of sample. Extensive resultsshow that our method outperforms all the state-of-the-art models on thechallenging NExT-QA benchmark, which demonstrates the effectiveness of theproposed method.

Rethinking Multi-Modal Alignment in Video Question Answering from Feature and Sample Perspectives