English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Training at the edge utilizes continuously evolving data generated atdifferent locations. Privacy concerns prohibit the co-location of thisspatially as well as temporally distributed data, deeming it crucial to designtraining algorithms that enable efficient continual learning over decentralizedprivate data. Decentralized learning allows serverless training with spatiallydistributed data. A fundamental barrier in such distributed learning is thehigh bandwidth cost of communicating model updates between agents. Moreover,existing works under this training paradigm are not inherently suitable forlearning a temporal sequence of tasks while retaining the previously acquiredknowledge. In this work, we propose CoDeC, a novel communication-efficientdecentralized continual learning algorithm which addresses these challenges. Wemitigate catastrophic forgetting while learning a task sequence in adecentralized learning setup by combining orthogonal gradient projection withgossip averaging across decentralized agents. Further, CoDeC includes a novellossless communication compression scheme based on the gradient subspaces. Weexpress layer-wise gradients as a linear combination of the basis vectors ofthese gradient subspaces and communicate the associated coefficients. Wetheoretically analyze the convergence rate for our algorithm and demonstratethrough an extensive set of experiments that CoDeC successfully learnsdistributed continual tasks with minimal forgetting. The proposed compressionscheme results in up to 4.8x reduction in communication costs withiso-performance as the full communication baseline.

CoDeC: Communication-Efficient Decentralized Continual Learning