English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

ViTs are often too computationally expensive to be fitted onto real-worldresource-constrained devices, due to (1) their quadratically increasedcomplexity with the number of input tokens and (2) their overparameterizedself-attention heads and model depth. In parallel, different images are ofvaried complexity and their different regions can contain various levels ofvisual information, indicating that treating all regions/tokens equally interms of model complexity is unnecessary while such opportunities for trimmingdown ViTs' complexity have not been fully explored. To this end, we propose aMulti-grained Input-adaptive Vision Transformer framework dubbed MIA-Formerthat can input-adaptively adjust the structure of ViTs at threecoarse-to-fine-grained granularities (i.e., model depth and the number of modelheads/tokens). In particular, our MIA-Former adopts a low-cost network trainedwith a hybrid supervised and reinforcement training method to skip unnecessarylayers, heads, and tokens in an input adaptive manner, reducing the overallcomputational cost. Furthermore, an interesting side effect of our MIA-Formeris that its resulting ViTs are naturally equipped with improved robustnessagainst adversarial attacks over their static counterparts, becauseMIA-Former's multi-grained dynamic control improves the model diversity similarto the effect of ensemble and thus increases the difficulty of adversarialattacks against all its sub-models. Extensive experiments and ablation studiesvalidate that the proposed MIA-Former framework can effectively allocatecomputation budgets adaptive to the difficulty of input images meanwhileincrease robustness, achieving state-of-the-art (SOTA) accuracy-efficiencytrade-offs, e.g., 20% computation savings with the same or even a higheraccuracy compared with SOTA dynamic transformer models.

MIA-Former: Efficient and Robust Vision Transformers via Multi-grained Input-Adaptation