English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Graph neural networks (GNNs) have become one of the most popular researchtopics in both academia and industry communities for their strong ability inhandling irregular graph data. However, large-scale datasets are posing greatchallenges for deploying GNNs in edge devices with limited resources and modelcompression techniques have drawn considerable research attention. Existingmodel compression techniques such as knowledge distillation (KD) mainly focuson convolutional neural networks (CNNs). Only limited attempts have been maderecently for distilling knowledge from GNNs in an offline manner. As theperformance of the teacher model does not necessarily improve as the number oflayers increases in GNNs, selecting an appropriate teacher model will requiresubstantial efforts. To address these challenges, we propose a novel onlineknowledge distillation framework called Alignahead++ in this paper.Alignahead++ transfers structure and feature information in a student layer tothe previous layer of another simultaneously trained student model in analternating training procedure. Meanwhile, to avoid over-smoothing problem inGNNs, deep supervision is employed in Alignahead++ by adding an auxiliaryclassifier in each intermediate layer to prevent the collapse of the nodefeature embeddings. Experimental results on four datasets including PPI, Cora,PubMed and CiteSeer demonstrate that the student performance is consistentlyboosted in our collaborative training framework without the supervision of apre-trained teacher model and its effectiveness can generally be improved byincreasing the number of students.

Online Cross-Layer Knowledge Distillation on Graph Neural Networks with Deep Supervision