English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Federated learning (FL) offers privacy-preserving decentralized machinelearning, optimizing models at edge clients without sharing private data.Simultaneously, foundation models (FMs) have gained traction in the artificialintelligence (AI) community due to their exceptional performance across varioustasks. However, integrating FMs into FL presents challenges, primarily due totheir substantial size and intensive resource requirements. This is especiallytrue when considering the resource heterogeneity in edge FL systems. We presentan adaptive framework for Resource-aware Federated Foundation Models (RaFFM) toaddress these challenges. RaFFM introduces specialized model compressionalgorithms tailored for FL scenarios, such as salient parameter prioritizationand high-performance subnetwork extraction. These algorithms enable dynamicscaling of given transformer-based FMs to fit heterogeneous resourceconstraints at the network edge during both FL's optimization and deploymentstages. Experimental results demonstrate that RaFFM shows significantsuperiority in resource utilization efficiency and uses fewer resources todeploy FMs to FL. Despite the lower resource consumption, target modelsoptimized by RaFFM achieve performance on par with traditional FL methodsapplied to full-sized FMs. This is evident across tasks in both naturallanguage processing and computer vision domains.

Bridging the Gap Between Foundation Models and Heterogeneous Federated Learning