English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Graphs are omnipresent and GNNs are a powerful family of neural networks forlearning over graphs. Despite their popularity, scaling GNNs either bydeepening or widening suffers from prevalent issues of unhealthy gradients,over-smoothening, information squashing, which often lead to sub-standardperformance. In this work, we are interested in exploring a principled way toscale GNNs capacity without deepening or widening, which can improve itsperformance across multiple small and large graphs. Motivated by the recentintriguing phenomenon of model soups, which suggest that fine-tuned weights ofmultiple large-language pre-trained models can be merged to a better minima, weargue to exploit the fundamentals of model soups to mitigate the aforementionedissues of memory bottleneck and trainability during GNNs scaling. Morespecifically, we propose not to deepen or widen current GNNs, but insteadpresent a data-centric perspective of model soups tailored for GNNs, i.e., tobuild powerful GNNs. By dividing giant graph data, we build multipleindependently and parallelly trained weaker GNNs (soup ingredient) without anyintermediate communication, and combine their strength using a greedyinterpolation soup procedure to achieve state-of-the-art performance. Comparedto concurrent distributed GNN training works such as Jiong et. al. 2023, wetrain each soup ingredient by sampling different subgraphs per epoch and theirrespective sub-models are merged only after being fully trained (rather thanintermediately so). Moreover, we provide a wide variety of model souppreparation techniques by leveraging state-of-the-art graph sampling and graphpartitioning approaches that can handle large graphs. Codes are available at:\url{https://github.com/VITA-Group/graph_ladling}.

Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication