English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained vision-language models (VLMs) have achieved impressive results ina range of vision-language tasks. However, popular VLMs usually consist ofhundreds of millions of parameters which brings challenges for fine-tuning anddeployment in real-world applications due to space, memory, and latencyconstraints. In this work, we introduce a distilling then pruning framework tocompress large vision-language models into smaller, faster, and more accurateones. We first shrink the size of a pre-trained large VLM and apply knowledgedistillation in the vision-language pre-training stage to obtain atask-agnostic compact VLM. Then we propose a modal-adaptive pruning algorithmto automatically infer the importance of vision and language modalities fordifferent downstream tasks and adaptively remove redundant structures andneurons in different encoders with controllable target sparsity. We apply ourframework to train EfficientVLM, a fast and accurate vision-language modelconsisting of 6 vision layers, 3 text layers, and 3 cross-modal fusion layers,accounting for only 93 million parameters in total, which is 44.3% of theteacher model. EfficientVLM retains 98.4% performance of the teacher model andaccelerates its inference speed by 2.2x. EfficientVLM achieves a large absoluteimprovement over previous SoTA efficient VLMs of similar sizes by a largemargin on various vision-language tasks, including VQAv2 (+4.9%), NLVR2(+5.6%), ITR (R@1 on TR +17.2%, on IR + 15.6% ) and COCO caption generation(CIDEr +6.5), demonstrating a large potential on training lightweight VLMs.

EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning