English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Systematic adaptation of network depths at runtime can be an effective way tocontrol inference latency and meet the resource condition of various devices.However, previous depth adaptive networks do not provide general principles anda formal explanation on why and which layers can be skipped, and, hence, theirapproaches are hard to be generalized and require long and complex trainingsteps. In this paper, we present an architectural pattern and training methodfor adaptive depth networks that can provide flexible accuracy-efficiencytrade-offs in a single network. In our approach, every residual stage isdivided into 2 consecutive sub-paths with different properties. While the firstsub-path is mandatory for hierarchical feature learning, the other is optimizedto incur minimal performance degradation even if it is skipped. Unlike previousadaptive networks, our approach does not iteratively self-distill a fixed setof sub-networks, resulting in significantly shorter training time. However,once deployed on devices, it can instantly construct sub-networks of varyingdepths to provide various accuracy-efficiency trade-offs in a single model. Weprovide a formal rationale for why the proposed architectural pattern andtraining method can reduce overall prediction errors while minimizing theimpact of skipping selected sub-paths. We also demonstrate the generality andeffectiveness of our approach with various residual networks, both fromconvolutional neural networks and vision transformers.

Adaptive Depth Networks with Skippable Sub-Paths