English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the wide and deep adoption of deep learning models in real applications,there is an increasing need to model and learn the representations of theneural networks themselves. These models can be used to estimate attributes ofdifferent neural network architectures such as the accuracy and latency,without running the actual training or inference tasks. In this paper, wepropose a neural architecture representation model that can be used to estimatethese attributes holistically. Specifically, we first propose a simple andeffective tokenizer to encode both the operation and topology information of aneural network into a single sequence. Then, we design a multi-stage fusiontransformer to build a compact vector representation from the convertedsequence. For efficient model training, we further propose an information flowconsistency augmentation and correspondingly design an architecture consistencyloss, which brings more benefits with less augmentation samples compared withprevious random augmentation strategies. Experiment results on NAS-Bench-101,NAS-Bench-201, DARTS search space and NNLQP show that our proposed frameworkcan be used to predict the aforementioned latency and accuracy attributes ofboth cell architectures and whole deep neural networks, and achieves promisingperformance. Code is available at https://github.com/yuny220/NAR-Former.

NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Prediction