English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent vision-language models have achieved tremendous progress far beyondwhat we ever expected. However, their computational costs are also dramaticallygrowing with rapid development, especially for the large models. It makes modelacceleration exceedingly critical in a scenario of limited resources. Althoughextensively studied for unimodal models, the acceleration for multimodalmodels, especially the vision-language Transformers, is relativelyunder-explored. To pursue more efficient and accessible vision-languageTransformers, this paper introduces \textbf{Cross}-\textbf{G}uided\textbf{E}nsemble of \textbf{T}okens (\textbf{\emph{CrossGET}}), a universalacceleration framework for vision-language Transformers. This frameworkadaptively combines tokens through real-time, cross-modal guidance, therebyachieving substantial acceleration while keeping high performance.\textit{CrossGET} has two key innovations: 1) \textit{Cross-Guided Matching andEnsemble}. \textit{CrossGET} incorporates cross-modal guided token matching andensemble to exploit cross-modal information effectively, only introducingcross-modal tokens with negligible extra parameters. 2) \textit{Complete-GraphSoft Matching}. In contrast to the existing bipartite soft matching approach,\textit{CrossGET} introduces a complete-graph soft matching policy to achievemore reliable token-matching results while maintaining parallelizability andhigh efficiency. Extensive experiments are conducted on various vision-languagetasks, including image-text retrieval, visual reasoning, image captioning, andvisual question answering. Performance on both classic multimodal architecturesand emerging multimodal LLMs demonstrate the effectiveness and versatility ofthe proposed \textit{CrossGET} framework. The code will be at\url{https://github.com/sdc17/CrossGET}.

CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers