English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While Graph Neural Networks (GNNs) are popular in the deep learningcommunity, they suffer from several challenges including over-smoothing,over-squashing, and gradient vanishing. Recently, a series of models haveattempted to relieve these issues by first augmenting the node features andthen imposing node-wise functions based on Multi-Layer Perceptron (MLP), whichare widely referred to as GA-MLP models. However, while GA-MLP models enjoydeeper architectures for better accuracy, their efficiency largelydeteriorates. Moreover, popular acceleration techniques such asstochastic-version or data-parallelism cannot be effectively applied due to thedependency among samples (i.e., nodes) in graphs. To address these issues, inthis paper, instead of data parallelism, we propose a parallel graph deeplearning Alternating Direction Method of Multipliers (pdADMM-G) framework toachieve model parallelism: parameters in each layer of GA-MLP models can beupdated in parallel. The extended pdADMM-G-Q algorithm reduces communicationcosts by introducing the quantization technique. Theoretical convergence to a(quantized) stationary point of the pdADMM-G algorithm and the pdADMM-G-Qalgorithm is provided with a sublinear convergence rate $o(1/k)$, where $k$ isthe number of iterations. Extensive experiments demonstrate the convergence oftwo proposed algorithms. Moreover, they lead to a more massive speedup andbetter performance than all state-of-the-art comparison methods on ninebenchmark datasets. Last but not least, the proposed pdADMM-G-Q algorithmreduces communication overheads by up to $45\%$ without loss of performance.Our code is available at \url{https://github.com/xianggebenben/pdADMM-G}.

Towards Quantized Model Parallelism for Graph-Augmented MLPs Based on Gradient-Free ADMM Framework