English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Conventional Transformer-based Video Question Answering (VideoQA) approachesgenerally encode frames independently through one or more image encodersfollowed by interaction between frames and question. However, such schema wouldincur significant memory use and inevitably slow down the training andinference speed. In this work, we present a highly efficient approach forVideoQA based on existing vision-language pre-trained models where weconcatenate video frames to a $n\times n$ matrix and then convert it to oneimage. By doing so, we reduce the use of the image encoder from $n^{2}$ to $1$while maintaining the temporal structure of the original video. Experimentalresults on MSRVTT and TrafficQA show that our proposed approach achievesstate-of-the-art performance with nearly $4\times$ faster speed and only 30%memory use. We show that by integrating our approach into VideoQA systems wecan achieve comparable, even superior, performance with a significant speed upfor training and inference. We believe the proposed approach can facilitateVideoQA-related research by reducing the computational requirements for thosewho have limited access to budgets and resources. Our code will be madepublicly available for research use.

Is a Video worth $n\times n$ Images? A Highly Efficient Approach to Transformer-based Video Question Answering