English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the increasing availability of Foundation Models, federated tuning hasgarnered attention in the field of federated learning, utilizing data andcomputation resources from multiple clients to collaboratively fine-tunefoundation models. However, in real-world federated scenarios, there oftenexist a multitude of heterogeneous clients with varying computation andcommunication resources, rendering them incapable of supporting the entiremodel fine-tuning process. In response to this challenge, we propose a novelfederated tuning algorithm, FedRA. The implementation of FedRA isstraightforward and can be seamlessly integrated into any transformer-basedmodel without the need for further modification to the original model.Specifically, in each communication round, FedRA randomly generates anallocation matrix. For resource-constrained clients, it reorganizes a smallnumber of layers from the original model based on the allocation matrix andfine-tunes using LoRA. Subsequently, the server aggregates the updated LoRAparameters from the clients according to the current allocation matrix into thecorresponding layers of the original model. It is worth noting that FedRA alsosupports scenarios where none of the clients can support the entire globalmodel, which is an impressive advantage. We conduct experiments on twolarge-scale image datasets, DomainNet and NICO++, under various non-iidsettings. The results demonstrate that FedRA outperforms the compared methodssignificantly. The source code is available at\url{https://github.com/leondada/FedRA}.

FedRA: A Random Allocation Strategy for Federated Tuning to Unleash the Power of Heterogeneous Clients