English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Continual Federated Learning (CFL) combines Federated Learning (FL), thedecentralized learning of a central model on a number of client devices thatmay not communicate their data, and Continual Learning (CL), the learning of amodel from a continual stream of data without keeping the entire history. InCL, the main challenge is \textit{forgetting} what was learned from past data.While replay-based algorithms that keep a small pool of past training data areeffective to reduce forgetting, only simple replay sample selection strategieshave been applied to CFL in prior work, and no previous work has exploredcoordination among clients for better sample selection. To bridge this gap, weadapt a replay sample selection objective based on loss gradient diversity toCFL and propose a new relaxation-based selection of samples to optimize theobjective. Next, we propose a practical algorithm to coordinate gradient-basedreplay sample selection across clients without communicating private data. Webenchmark our coordinated and uncoordinated replay sample selection algorithmsagainst random sampling-based baselines with language models trained on a largescale de-identified real-world text dataset. We show that gradient-based sampleselection methods both boost performance and reduce forgetting compared torandom sampling methods, with our coordination method showing gains early inthe low replay size regime (when the budget for storing past data is small).

Coordinated Replay Sample Selection for Continual Federated Learning