English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large-scale Transformer models bring significant improvements for variousdownstream vision language tasks with a unified architecture. The performanceimprovements come with increasing model size, resulting in slow inference speedand increased cost for severing. While some certain predictions benefit fromthe full complexity of the large-scale model, not all of inputs need the sameamount of computation to conduct, potentially leading to computation resourcewaste. To handle this challenge, early exiting is proposed to adaptivelyallocate computational power in term of input complexity to improve inferenceefficiency. The existing early exiting strategies usually adopt outputconfidence based on intermediate layers as a proxy of input complexity to incurthe decision of skipping following layers. However, such strategies cannotapply to encoder in the widely-used unified architecture with both encoder anddecoder due to difficulty of output confidence estimation in the encoder. It issuboptimal in term of saving computation power to ignore the early exiting inencoder component. To handle this challenge, we propose a novel early exitingstrategy for unified visual language models, which allows dynamically skip thelayers in encoder and decoder simultaneously in term of input layer-wisesimilarities with multiple times of early exiting, namely \textbf{MuE}. Bydecomposing the image and text modalities in the encoder, MuE is flexible andcan skip different layers in term of modalities, advancing the inferenceefficiency while minimizing performance drop. Experiments on the SNLI-VE and MSCOCO datasets show that the proposed approach MuE can reduce expected inferencetime by up to 50\% and 40\% while maintaining 99\% and 96\% performancerespectively.

You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model