English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite the enormous success of Graph Convolutional Networks (GCNs) inmodeling graph-structured data, most of the current GCNs are shallow due to thenotoriously challenging problems of over-smoothening and information squashingalong with conventional difficulty caused by vanishing gradients andover-fitting. Previous works have been primarily focused on the study ofover-smoothening and over-squashing phenomena in training deep GCNs.Surprisingly, in comparison with CNNs/RNNs, very limited attention has beengiven to understanding how healthy gradient flow can benefit the trainabilityof deep GCNs. In this paper, firstly, we provide a new perspective of gradientflow to understand the substandard performance of deep GCNs and hypothesizethat by facilitating healthy gradient flow, we can significantly improve theirtrainability, as well as achieve state-of-the-art (SOTA) level performance fromvanilla-GCNs. Next, we argue that blindly adopting the Glorot initializationfor GCNs is not optimal, and derive a topology-aware isometric initializationscheme for vanilla-GCNs based on the principles of isometry. Additionally,contrary to ad-hoc addition of skip-connections, we propose to usegradient-guided dynamic rewiring of vanilla-GCNs} with skip connections. Ourdynamic rewiring method uses the gradient flow within each layer duringtraining to introduce on-demand skip-connections adaptively. We provideextensive empirical evidence across multiple datasets that our methods improvegradient flow in deep vanilla-GCNs and significantly boost their performance tocomfortably compete and outperform many fancy state-of-the-art methods. Codesare available at: https://github.com/VITA-Group/GradientGCN.

Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again