English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent advances in large video-language models have displayed promisingoutcomes in video comprehension. Current approaches straightforwardly convertvideo into language tokens and employ large language models for multi-modaltasks. However, this method often leads to the generation of irrelevantcontent, commonly known as "hallucination", as the length of the text increasesand the impact of the video diminishes. To address this problem, we proposeVista-LLaMA, a novel framework that maintains the consistent distance betweenall visual tokens and any language tokens, irrespective of the generated textlength. Vista-LLaMA omits relative position encoding when determining attentionweights between visual and text tokens, retaining the position encoding fortext and text tokens. This amplifies the effect of visual tokens on textgeneration, especially when the relative distance is longer between visual andtext tokens. The proposed attention mechanism significantly reduces the chanceof producing irrelevant text related to the video content. Furthermore, wepresent a sequential visual projector that projects the current video frameinto tokens of language space with the assistance of the previous frame. Thisapproach not only captures the temporal relationship within the video, but alsoallows less visual tokens to encompass the entire video. Our approachsignificantly outperforms various previous methods (e.g., Video-ChatGPT,MovieChat) on four challenging open-ended video question answering benchmarks.We reach an accuracy of 60.7 on the zero-shot NExT-QA and 60.5 on the zero-shotMSRVTT-QA, setting a new state-of-the-art performance. This project isavailable at https://jinxxian.github.io/Vista-LLaMA.

Vista-LLaMA: Reliable Video Narrator via Equal Distance to Visual Tokens