English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the acceleration of deep neural network training, the GPU has become themainstream platform. GPUs face substantial challenges on GNNs, such as workloadimbalance and memory access irregularities, leading to underutilized hardware.Existing solutions such as PyG, DGL with cuSPARSE, and GNNAdvisor frameworkspartially address these challenges but memory traffic is still significant.  We argue that drastic performance improvements can only be achieved by thevertical optimization of algorithm and system innovations, rather than treatingthe speedup optimization as an "after-thought" (i.e., (i) given a GNNalgorithm, designing an accelerator, or (ii) given hardware, mainly optimizingthe GNN algorithm). In this paper, we present MaxK-GNN, an advancedhigh-performance GPU training system integrating algorithm and systeminnovation. (i) We introduce the MaxK nonlinearity and provide a theoreticalanalysis of MaxK nonlinearity as a universal approximator, and present theCompressed Balanced Sparse Row (CBSR) format, designed to store the data andindex of the feature matrix after nonlinearity; (ii) We design a coalescingenhanced forward computation with row-wise product-based SpGEMM Kernel usingCBSR for input feature matrix fetching and strategic placement of a sparseoutput accumulation buffer in shared memory; (iii) We develop an optimizedbackward computation with outer product-based and SSpMM Kernel.  We conduct extensive evaluations of MaxK-GNN and report the end-to-end systemrun-time. Experiments show that MaxK-GNN system could approach the theoreticalspeedup limit according to Amdahl's law. We achieve comparable accuracy to SOTAGNNs, but at a significantly increased speed: 3.22/4.24 times speedup (vs.theoretical limits, 5.52/7.27 times) on Reddit compared to DGL and GNNAdvisorimplementations.

MaxK-GNN: Towards Theoretical Speed Limits for Accelerating Graph Neural Networks Training