English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent years have witnessed the great success of Graph Neural Networks (GNNs)in handling graph-related tasks. However, MLPs remain the primary workhorse forpractical industrial applications due to their desirable inference efficiencyand scalability. To reduce their gaps, one can directly distill knowledge froma well-designed teacher GNN to a student MLP, which is termed as GNN-to-MLPdistillation. However, the process of distillation usually entails a loss ofinformation, and ``which knowledge patterns of GNNs are more likely to be leftand distilled into MLPs?" becomes an important question. In this paper, wefirst factorize the knowledge learned by GNNs into low- and high-frequencycomponents in the spectral domain and then derive their correspondence in thespatial domain. Furthermore, we identified a potential information drowningproblem for existing GNN-to-MLP distillation, i.e., the high-frequencyknowledge of the pre-trained GNNs may be overwhelmed by the low-frequencyknowledge during distillation; we have described in detail what it represents,how it arises, what impact it has, and how to deal with it. In this paper, wepropose an efficient Full-Frequency GNN-to-MLP (FF-G2M) distillation framework,which extracts both low-frequency and high-frequency knowledge from GNNs andinjects it into MLPs. Extensive experiments show that FF-G2M improves over thevanilla MLPs by 12.6% and outperforms its corresponding teacher GNNs by 2.6%averaged over six graph datasets and three common GNN architectures.

Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Framework