English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Recent years have seen considerable progress in the continual training ofdeep neural networks, predominantly thanks to approaches that add replay orregularization terms to the loss function to approximate the joint loss overall tasks so far. However, we show that even with a perfect approximation tothe joint loss, these approaches still suffer from temporary but substantialforgetting when starting to train on a new task. Motivated by this 'stabilitygap', we propose that continual learning strategies should focus not only onthe optimization objective, but also on the way this objective is optimized.While there is some continual learning work that alters the optimizationtrajectory (e.g., using gradient projection techniques), this line of researchis positioned as alternative to improving the optimization objective, while weargue it should be complementary. To evaluate the merits of our proposition, weplan to combine replay-approximated joint objectives with gradientprojection-based optimization routines to test whether the addition of thelatter provides benefits in terms of (1) alleviating the stability gap, (2)increasing the learning efficiency and (3) improving the final learningoutcome.

Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How