English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Subgraph-wise sampling -- a promising class of mini-batch training techniquesfor graph neural networks (GNNs -- is critical for real-world applications.During the message passing (MP) in GNNs, subgraph-wise sampling methods discardmessages outside the mini-batches in backward passes to avoid the well-knownneighbor explosion problem, i.e., the exponentially increasing dependencies ofnodes with the number of MP iterations. However, discarding messages maysacrifice the gradient estimation accuracy, posing significant challenges totheir convergence analysis and convergence speeds. To address this challenge,we propose a novel subgraph-wise sampling method with a convergence guarantee,namely Local Message Compensation (LMC). To the best of our knowledge, LMC isthe first subgraph-wise sampling method with provable convergence. The key ideais to retrieve the discarded messages in backward passes based on a messagepassing formulation of backward passes. By efficient and effectivecompensations for the discarded messages in both forward and backward passes,LMC computes accurate mini-batch gradients and thus accelerates convergence.Moreover, LMC is applicable to various MP-based GNN architectures, includingconvolutional GNNs (finite message passing iterations with different layers)and recurrent GNNs (infinite message passing iterations with a shared layer).Experiments on large-scale benchmarks demonstrate that LMC is significantlyfaster than state-of-the-art subgraph-wise sampling methods.

Provably Convergent Subgraph-wise Sampling for Fast GNN Training