English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Continual learning is inherently a constrained learning problem. The goal isto learn a predictor under a \emph{no-forgetting} requirement. Although severalprior studies formulate it as such, they do not solve the constrained problemexplicitly. In this work, we show that it is both possible and beneficial toundertake the constrained optimization problem directly. To do this, weleverage recent results in constrained learning through Lagrangian duality. Wefocus on memory-based methods, where a small subset of samples from previoustasks can be stored in a replay buffer. In this setting, we analyze twoversions of the continual learning problem: a coarse approach with constraintsat the task level and a fine approach with constraints at the sample level. Weshow that dual variables indicate the sensitivity of the optimal value withrespect to constraint perturbations. We then leverage this result to partitionthe buffer in the coarse approach, allocating more resources to harder tasks,and to populate the buffer in the fine approach, including only impactfulsamples. We derive sub-optimality bounds, and empirically corroborate ourtheoretical results in various continual learning benchmarks. We also discussthe limitations of these methods with respect to the amount of memory availableand the number of constraints involved in the optimization problem.

Primal-Dual Continual Learning: Stability and Plasticity through Lagrange Multipliers