English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With increasing privacy concerns on data, recent studies have madesignificant progress using federated learning (FL) on privacy-sensitive naturallanguage processing (NLP) tasks. Much literature suggests fully fine-tuningpre-trained language models (PLMs) in the FL paradigm can mitigate the dataheterogeneity problem and close the performance gap with centralized training.However, large PLMs bring the curse of prohibitive communication overhead andlocal model adaptation costs for the FL system. To this end, we introducevarious parameter-efficient tuning (PETuning) methods into federated learning.Specifically, we provide a holistic empirical study of representative PLMstuning methods in FL. The experimental results cover the analysis of dataheterogeneity levels, data scales, and different FL scenarios. Overallcommunication overhead can be significantly reduced by locally tuning andglobally aggregating lightweight model parameters while maintaining acceptableperformance in various FL settings. To facilitate the research of PETuning inFL, we also develop a federated tuning framework FedPETuning, which allowspractitioners to exploit different PETuning methods under the FL trainingparadigm conveniently. The source code is available at\url{https://github.com/iezhuozhuo/FedETuning/tree/deltaTuning}.

When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods