English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural Architecture Search (NAS) has become a de facto approach in the recenttrend of AutoML to design deep neural networks (DNNs). Efficient ornear-zero-cost NAS proxies are further proposed to address the demandingcomputational issues of NAS, where each candidate architecture network onlyrequires one iteration of backpropagation. The values obtained from the proxiesare considered the predictions of architecture performance on downstream tasks.However, two significant drawbacks hinder the extended usage of Efficient NASproxies. (1) Efficient proxies are not adaptive to various search spaces. (2)Efficient proxies are not extensible to multi-modality downstream tasks. Basedon the observations, we design a Extensible proxy (Eproxy) that utilizesself-supervised, few-shot training (i.e., 10 iterations of backpropagation)which yields near-zero costs. The key component that makes Eproxy efficient isan untrainable convolution layer termed barrier layer that add thenon-linearities to the optimization spaces so that the Eproxy can discriminatethe performance of architectures in the early stage. Furthermore, to makeEproxy adaptive to different downstream tasks/search spaces, we propose aDiscrete Proxy Search (DPS) to find the optimized training settings for Eproxywith only handful of benchmarked architectures on the target tasks. Ourextensive experiments confirm the effectiveness of both Eproxy and Eproxy+DPS.Code is available at https://github.com/leeyeehoo/GenNAS-Zero.

Extensible Proxy for Efficient NAS