English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent research has demonstrated the efficacy of pre-training graph neuralnetworks (GNNs) to capture the transferable graph semantics and enhance theperformance of various downstream tasks. However, the semantic knowledgelearned from pretext tasks might be unrelated to the downstream task, leadingto a semantic gap that limits the application of graph pre-training. To reducethis gap, traditional approaches propose hybrid pre-training to combine variouspretext tasks together in a multi-task learning fashion and learn multi-grainedknowledge, which, however, cannot distinguish tasks and results in sometransferable task-specific knowledge distortion by each other. Moreover, mostGNNs cannot distinguish nodes located in different parts of the graph, makingthem fail to learn position-specific knowledge and lead to suboptimalperformance. In this work, inspired by the prompt-based tuning in naturallanguage processing, we propose a unified framework for graph hybridpre-training which injects the task identification and position identificationinto GNNs through a prompt mechanism, namely multi-task graph dual prompt(ULTRA-DP). Based on this framework, we propose a prompt-based transferabilitytest to find the most relevant pretext task in order to reduce the semanticgap. To implement the hybrid pre-training tasks, beyond the classical edgeprediction task (node-node level), we further propose a novel pre-trainingparadigm based on a group of $k$-nearest neighbors (node-group level). Thecombination of them across different scales is able to comprehensively expressmore structural semantics and derive richer multi-grained knowledge. Extensiveexperiments show that our proposed ULTRA-DP can significantly enhance theperformance of hybrid pre-training methods and show the generalizability toother pre-training tasks and backbone architectures.

ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt