English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The message-passing mechanism helps Graph Neural Networks (GNNs) achieveremarkable results on various node classification tasks. Nevertheless, therecursive nodes fetching and aggregation in message-passing cause inferencelatency when deploying GNNs to large-scale graphs. One promising inferenceacceleration direction is to distill the GNNs into message-passing-free studentmulti-layer perceptrons (MLPs). However, the MLP student cannot fully learn thestructure knowledge due to the lack of structure inputs, which causes inferiorperformance in the heterophily and inductive scenarios. To address this, weintend to inject structure information into MLP-like students in low-latencyand interpretable ways. Specifically, we first design a Structure-Aware MLP(SA-MLP) student that encodes both features and structures withoutmessage-passing. Then, we introduce a novel structure-mixing knowledgedistillation strategy to enhance the learning ability of MLPs for structureinformation. Furthermore, we design a latent structure embedding approximationtechnique with two-stage distillation for inductive scenarios. Extensiveexperiments on eight benchmark datasets under both transductive and inductivesettings show that our SA-MLP can consistently outperform the teacher GNNs,while maintaining faster inference as MLPs. The source code of our work can befound in https://github.com/JC-202/SA-MLP.

SA-MLP: Distilling Graph Knowledge from GNNs into Structure-Aware MLP