English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural Architecture Search (NAS) has emerged as a favoured method forunearthing effective neural architectures. Recent development of large modelshas intensified the demand for faster search speeds and more accurate searchresults. However, designing large models by NAS is challenging due to thedramatical increase of search space and the associated huge performanceevaluation cost. Consider a typical modular search space widely used in NAS, inwhich a neural architecture consists of $m$ block nodes and a block node has$n$ alternative blocks. Facing the space containing $n^m$ candidate networks,existing NAS methods attempt to find the best one by searching and evaluatingcandidate networks directly.Different from the general strategy that takesarchitecture search as a whole problem, we propose a novel divide-and-conquerstrategy by making use of the modular nature of the search space.Here, weintroduce MathNAS, a general NAS framework based on mathematical programming.InMathNAS, the performances of the $m*n$ possible building blocks in the searchspace are calculated first, and then the performance of a network is directlypredicted based on the performances of its building blocks. Although estimatingblock performances involves network training, just as what happens for networkperformance evaluation in existing NAS methods, predicting network performanceis completely training-free and thus extremely fast. In contrast to the $n^m$candidate networks to evaluate in existing NAS methods, which require trainingand a formidable computational burden, there are only $m*n$ possible blocks tohandle in MathNAS. Therefore, our approach effectively reduces the complexityof network performance evaluation.Our code is available athttps://github.com/wangqinsi1/MathNAS.

MathNAS: If Blocks Have a Role in Mathematical Architecture Design