English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

It has become a popular paradigm to transfer the knowledge of large-scalepre-trained models to various downstream tasks via fine-tuning the entire modelparameters. However, with the growth of model scale and the rising number ofdownstream tasks, this paradigm inevitably meets the challenges in terms ofcomputation consumption and memory footprint issues. Recently,Parameter-Efficient Fine-Tuning (PEFT) (e.g., Adapter, LoRA, BitFit) shows apromising paradigm to alleviate these concerns by updating only a portion ofparameters. Despite these PEFTs having demonstrated satisfactory performance innatural language processing, it remains under-explored for the question ofwhether these techniques could be transferred to graph-based tasks with GraphTransformer Networks (GTNs). Therefore, in this paper, we fill this gap byproviding extensive benchmarks with traditional PEFTs on a range of graph-baseddownstream tasks. Our empirical study shows that it is sub-optimal to directlytransfer existing PEFTs to graph-based tasks due to the issue of featuredistribution shift. To address this issue, we propose a novel structure-awarePEFT approach, named G-Adapter, which leverages graph convolution operation tointroduce graph structure (e.g., graph adjacent matrix) as an inductive bias toguide the updating process. Besides, we propose Bregman proximal pointoptimization to further alleviate feature distribution shift by preventing themodel from aggressive update. Extensive experiments demonstrate that G-Adapterobtains the state-of-the-art performance compared to the counterparts on ninegraph benchmark datasets based on two pre-trained GTNs, and delivers tremendousmemory footprint efficiency compared to the conventional paradigm.

G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks