English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural Architecture Search has attracted increasing attention in recentyears. Among them, differential NAS approaches such as DARTS, have gainedpopularity for the search efficiency. However, they still suffer from threemain issues, that are, the weak stability due to the performance collapse, thepoor generalization ability of the searched architectures, and the inferiorrobustness to different kinds of proxies. To solve the stability andgeneralization problems, a simple-but-effective regularization method, termedas Beta-Decay, is proposed to regularize the DARTS-based NAS searching process(i.e., $\beta$-DARTS). Specifically, Beta-Decay regularization can imposeconstraints to keep the value and variance of activated architecture parametersfrom being too large, thereby ensuring fair competition among architectureparameters and making the supernet less sensitive to the impact of input on theoperation set. In-depth theoretical analyses on how it works and why it worksare provided. Comprehensive experiments validate that Beta-Decay regularizationcan help to stabilize the searching process and makes the searched network moretransferable across different datasets. To address the robustness problem, wefirst benchmark different NAS methods under a wide range of proxy data, proxychannels, proxy layers and proxy epochs, since the robustness of NAS underdifferent kinds of proxies has not been explored before. We then conclude someinteresting findings and find that $\beta$-DARTS always achieves the bestresult among all compared NAS methods under almost all proxies. We furtherintroduce the novel flooding regularization to the weight optimization of$\beta$-DARTS (i.e., Bi-level regularization), and experimentally andtheoretically verify its effectiveness for improving the proxy robustness ofdifferentiable NAS.

$\beta$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search