English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Video Question Answering (VideoQA) aims to answer natural language questionsbased on the information observed in videos. Despite the recent success ofLarge Multimodal Models (LMMs) in image-language understanding and reasoning,they deal with VideoQA insufficiently by simply taking uniformly sampled framesas visual inputs, which ignores question-relevant visual clues. Moreover, thereare no human annotations for question-critical timestamps in existing VideoQAdatasets. In light of this, we propose a novel weakly supervised framework toenforce the LMMs to reason out the answers with question-critical moments asvisual inputs. Specifically, we fuse the question and answer pairs as eventdescriptions to find multiple keyframes as target moments, which will bepseudo-labels. With these pseudo-labels as additionally weak supervision, wedevise a lightweight Gaussian-based Contrastive Grounding (GCG) module. GCGlearns multiple Gaussian functions to characterize the temporal structure ofthe video, and sample question-critical frames as positive moments to be thevisual inputs of LMMs. Extensive experiments on several VideoQA benchmarksverify the effectiveness of our framework, and we achieve substantialimprovements compared to previous state-of-the-art methods.

Weakly Supervised Gaussian Contrastive Grounding with Large Multimodal Models for Video Question Answering