English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Foundation Models (FMs), such as LLaMA, BERT, GPT, ViT, and CLIP, havedemonstrated remarkable success in a wide range of applications, driven bytheir ability to leverage vast amounts of data for pre-training. However,optimizing FMs often requires access to sensitive data, raising privacyconcerns and limiting their applicability in many domains. In this paper, wepropose the Federated Foundation Models (FFMs) paradigm, which combines thebenefits of FMs and Federated Learning (FL) to enable privacy-preserving andcollaborative learning across multiple end-users. We discuss the potentialbenefits and challenges of integrating FL into the lifespan of FMs, coveringpre-training, fine-tuning, and application. We further outline potential futureresearch avenues in FFM, including FFM pre-training, FFM fine-tuning, andfederated prompt tuning, which allow the development of more personalized andcontext-aware models while ensuring data privacy. Moreover, we explore thepossibility of continual/lifelong learning in FFMs, as increased computationalpower at the edge may unlock the potential for optimizing FMs using newlygenerated private data close to the data source. The proposed FFM conceptsoffer a flexible and scalable framework for training large language models in aprivacy-preserving manner, setting the stage for subsequent advancements inboth FM training and federated learning.

Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models