English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Continual learning is inherently a constrained learning problem. The goal isto learn a predictor under a no-forgetting requirement. Although several priorstudies formulate it as such, they do not solve the constrained problemexplicitly. In this work, we show that it is both possible and beneficial toundertake the constrained optimization problem directly. To do this, weleverage recent results in constrained learning through Lagrangian duality. Wefocus on memory-based methods, where a small subset of samples from previoustasks can be stored in a replay buffer. In this setting, we analyze twoversions of the continual learning problem: a coarse approach with constraintsat the task level and a fine approach with constraints at the sample level. Weshow that dual variables indicate the sensitivity of the optimal value of thecontinual learning problem with respect to constraint perturbations. We thenleverage this result to partition the buffer in the coarse approach, allocatingmore resources to harder tasks, and to populate the buffer in the fineapproach, including only impactful samples. We derive a deviation bound on dualvariables as sensitivity indicators, and empirically corroborate this result indiverse continual learning benchmarks. We also discuss the limitations of thesemethods with respect to the amount of memory available and the expressivenessof the parametrization.

Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocation