English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper proposes Video-Teller, a video-language foundation model thatleverages multi-modal fusion and fine-grained modality alignment tosignificantly enhance the video-to-text generation task. Video-Teller booststhe training efficiency by utilizing frozen pretrained vision and languagemodules. It capitalizes on the robust linguistic capabilities of large languagemodels, enabling the generation of both concise and elaborate videodescriptions. To effectively integrate visual and auditory information,Video-Teller builds upon the image-based BLIP-2 model and introduces a cascadedQ-Former which fuses information across frames and ASR texts. To better guidevideo summarization, we introduce a fine-grained modality alignment objective,where the cascaded Q-Former's output embedding is trained to align with thecaption/summary embedding created by a pretrained text auto-encoder.Experimental results demonstrate the efficacy of our proposed video-languagefoundation model in accurately comprehending videos and generating coherent andprecise language descriptions. It is worth noting that the fine-grainedalignment enhances the model's capabilities (4% improvement of CIDEr score onMSR-VTT) with only 13% extra parameters in training and zero additional cost ininference.

Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling