English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The message passing-based graph neural networks (GNNs) have achieved greatsuccess in many real-world applications. However, training GNNs on large-scalegraphs suffers from the well-known neighbor explosion problem, i.e., theexponentially increasing dependencies of nodes with the number of messagepassing layers. Subgraph-wise sampling methods -- a promising class ofmini-batch training techniques -- discard messages outside the mini-batches inbackward passes to avoid the neighbor explosion problem at the expense ofgradient estimation accuracy. This poses significant challenges to theirconvergence analysis and convergence speeds, which seriously limits theirreliable real-world applications. To address this challenge, we propose a novelsubgraph-wise sampling method with a convergence guarantee, namely LocalMessage Compensation (LMC). To the best of our knowledge, LMC is the {\itfirst} subgraph-wise sampling method with provable convergence. The key idea ofLMC is to retrieve the discarded messages in backward passes based on a messagepassing formulation of backward passes. By efficient and effectivecompensations for the discarded messages in both forward and backward passes,LMC computes accurate mini-batch gradients and thus accelerates convergence. Wefurther show that LMC converges to first-order stationary points of GNNs.Experiments on large-scale benchmark tasks demonstrate that LMC significantlyoutperforms state-of-the-art subgraph-wise sampling methods in terms ofefficiency.

LMC: Fast Training of GNNs via Subgraph Sampling with Provable Convergence