English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Continual learning (CL) is a setting in which an agent has to learn from anincoming stream of data sequentially. CL performance evaluates the model'sability to continually learn and solve new problems with incremental availableinformation over time while retaining previous knowledge. Despite the numerousprevious solutions to bypass the catastrophic forgetting (CF) of previouslyseen tasks during the learning process, most of them still suffer significantforgetting, expensive memory cost, or lack of theoretical understanding ofneural networks' conduct while learning new tasks. While the issue that CLperformance degrades under different training regimes has been extensivelystudied empirically, insufficient attention has been paid from a theoreticalangle. In this paper, we establish a probabilistic framework to analyzeinformation flow through layers in networks for task sequences and its impacton learning performance. Our objective is to optimize the informationpreservation between layers while learning new tasks to manage task-specificknowledge passing throughout the layers while maintaining model performance onprevious tasks. In particular, we study CL performance's relationship withinformation flow in the network to answer the question "How can knowledge ofinformation flow between layers be used to alleviate CF?". Our analysisprovides novel insights of information adaptation within the layers during theincremental task learning process. Through our experiments, we provideempirical evidence and practically highlight the performance improvement acrossmultiple tasks.

Theoretical Understanding of the Information Flow on Continual Learning Performance