English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

State-of-the-art deep learning models for computer vision tasks are based onthe transformer architecture and often deployed in real-time applications. Inthis scenario, the resources available for every inference can vary, so it isuseful to be able to dynamically adapt execution to trade accuracy forefficiency. To create dynamic models, we leverage the resilience of visiontransformers to pruning and switch between different scaled versions of amodel. Surprisingly, we find that most FLOPs are generated by convolutions, notattention. These relative FLOP counts are not a good predictor of GPUperformance since GPUs have special optimizations for convolutions. Some modelsare fairly resilient and their model execution can be adapted withoutretraining, while all models achieve better accuracy with retrainingalternative execution paths. These insights mean that we can leverage CNNaccelerators and these alternative execution paths to enable efficient anddynamic vision transformer inference. Our analysis shows that leveraging thistype of dynamic execution can lead to saving 28\% of energy with a 1.4\%accuracy drop for SegFormer (63 GFLOPs), with no additional training, and 53\%of energy for ResNet-50 (4 GFLOPs) with a 3.3\% accuracy drop by switchingbetween pretrained Once-For-All models.

Vision Transformer Computation and Resilience for Dynamic Inference