English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recent years, neural networks have demonstrated an outstanding ability toachieve complex learning tasks across various domains. However, they sufferfrom the "catastrophic forgetting" problem when they face a sequence oflearning tasks, where they forget the old ones as they learn new tasks. Thisproblem is also highly related to the "stability-plasticity dilemma". The moreplastic the network, the easier it can learn new tasks, but the faster it alsoforgets previous ones. Conversely, a stable network cannot learn new tasks asfast as a very plastic network. However, it is more reliable to preserve theknowledge it has learned from the previous tasks. Several solutions have beenproposed to overcome the forgetting problem by making the neural networkparameters more stable, and some of them have mentioned the significance ofdropout in continual learning. However, their relationship has not beensufficiently studied yet. In this paper, we investigate this relationship andshow that a stable network with dropout learns a gating mechanism such that fordifferent tasks, different paths of the network are active. Our experimentsshow that the stability achieved by this implicit gating plays a very criticalrole in leading to performance comparable to or better than other involvedcontinual learning algorithms to overcome catastrophic forgetting.

Dropout as an Implicit Gating Mechanism For Continual Learning