English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

One-shot Neural architecture search (One-shot NAS) has been proposed as atime-efficient approach to obtain optimal subnet architectures and weightsunder different complexity cases by training only once. However, the subnetperformance obtained by weight sharing is often inferior to the performanceachieved by retraining. In this paper, we investigate the performance gap andattribute it to the use of uniform sampling, which is a common approach insupernet training. Uniform sampling concentrates training resources on subnetswith intermediate computational resources, which are sampled with highprobability. However, subnets with different complexity regions requiredifferent optimal training strategies for optimal performance. To address theproblem of uniform sampling, we propose ShiftNAS, a method that can adjust thesampling probability based on the complexity of subnets. We achieve this byevaluating the performance variation of subnets with different complexity anddesigning an architecture generator that can accurately and efficiently providesubnets with the desired complexity. Both the sampling probability and thearchitecture generator can be trained end-to-end in a gradient-based manner.With ShiftNAS, we can directly obtain the optimal model architecture andparameters for a given computational complexity. We evaluate our approach onmultiple visual network models, including convolutional neural networks (CNNs)and vision transformers (ViTs), and demonstrate that ShiftNAS ismodel-agnostic. Experimental results on ImageNet show that ShiftNAS can improvethe performance of one-shot NAS without additional consumption. Source codesare available at https://github.com/bestfleer/ShiftNAS.

ShiftNAS: Improving One-shot NAS via Probability Shift