English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Audio-Visual Question Answering (AVQA) task aims to answer questions aboutdifferent visual objects, sounds, and their associations in videos. Suchnaturally multi-modal videos are composed of rich and complex dynamicaudio-visual components, where most of which could be unrelated to the givenquestions, or even play as interference in answering the content of interest.Oppositely, only focusing on the question-aware audio-visual content could getrid of influence, meanwhile enabling the model to answer more efficiently. Inthis paper, we propose a Progressive Spatio-Temporal Perception Network(PSTP-Net), which contains three modules that progressively identify keyspatio-temporal regions w.r.t. questions. Specifically, a temporal segmentselection module is first introduced to select the most relevant audio-visualsegments related to the given question. Then, a spatial region selection moduleis utilized to choose the most relevant regions associated with the questionfrom the selected temporal segments. To further refine the selection offeatures, an audio-guided visual attention module is employed to perceive theassociation between auido and selected spatial regions. Finally, thespatio-temporal features from these modules are integrated for answering thequestion. Extensive experimental results on the public MUSIC-AVQA and AVQAdatasets provide compelling evidence of the effectiveness and efficiency ofPSTP-Net. Code is available at:\href{https://github.com/GeWu-Lab/PSTP-Net}{https://github.com/GeWu-Lab/PSTP-Net}

Progressive Spatio-temporal Perception for Audio-Visual Question Answering