English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent parameter-efficient finetuning (PEFT) techniques aim to improve overthe considerable cost of fully finetuning large pretrained language models(PLM). As different PEFT techniques proliferate, it is becoming difficult tocompare them, in particular in terms of (i) the structure and functionalitythey add to the PLM, (ii) the different types and degrees of efficiencyimprovements achieved, (iii) performance at different downstream tasks, and(iv) how differences in structure and functionality relate to efficiency andtask performance. To facilitate such comparisons, this paper presents areference architecture which standardises aspects shared by different PEFTtechniques, while isolating differences to specific locations and interactionswith the standard components. Through this process of standardising andisolating differences, a modular view of PEFT techniques emerges, supportingnot only direct comparison of different techniques and their efficiency andtask performance, but also systematic exploration of reusability andcomposability of the different types of finetuned modules. We demonstrate howthe reference architecture can be applied to understand properties and relativeadvantages of PEFT techniques, hence to inform selection of techniques forspecific tasks, and design choices for new PEFT techniques.

PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques