English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

As more deep learning models are being applied in real-world applications,there is a growing need for modeling and learning the representations of neuralnetworks themselves. An efficient representation can be used to predict targetattributes of networks without the need for actual training and deploymentprocedures, facilitating efficient network deployment and design. Recently,inspired by the success of Transformer, some Transformer-based representationlearning frameworks have been proposed and achieved promising performance inhandling cell-structured models. However, graph neural network (GNN) basedapproaches still dominate the field of learning representation for the entirenetwork. In this paper, we revisit Transformer and compare it with GNN toanalyse their different architecture characteristics. We then propose amodified Transformer-based universal neural network representation learningmodel NAR-Former V2. It can learn efficient representations from bothcell-structured networks and entire networks. Specifically, we first take thenetwork as a graph and design a straightforward tokenizer to encode the networkinto a sequence. Then, we incorporate the inductive representation learningcapability of GNN into Transformer, enabling Transformer to generalize betterwhen encountering unseen architecture. Additionally, we introduce a series ofsimple yet effective modifications to enhance the ability of the Transformer inlearning representation from graph structures. Our proposed method surpassesthe GNN-based method NNLP by a significant margin in latency estimation on theNNLQP dataset. Furthermore, regarding accuracy prediction on the NASBench101and NASBench201 datasets, our method achieves highly comparable performance toother state-of-the-art methods.

NAR-Former V2: Rethinking Transformer for Universal Neural Network Representation Learning