English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Deep Neural Networks (DNN) could forget the knowledge about earlier taskswhen learning new tasks, and this is known as \textit{catastrophic forgetting}.While recent continual learning methods are capable of alleviating thecatastrophic problem on toy-sized datasets, some issues still remain to betackled when applying them in real-world problems. Recently, the fastmask-based learning method (e.g. piggyback \cite{mallya2018piggyback}) isproposed to address these issues by learning only a binary element-wise mask ina fast manner, while keeping the backbone model fixed. However, the binary maskhas limited modeling capacity for new tasks. A more recent work\cite{hung2019compacting} proposes a compress-grow-based method (CPG) toachieve better accuracy for new tasks by partially training backbone model, butwith order-higher training cost, which makes it infeasible to be deployed intopopular state-of-the-art edge-/mobile-learning. The primary goal of this workis to simultaneously achieve fast and high-accuracy multi task adaption incontinual learning setting. Thus motivated, we propose a new training methodcalled \textit{kernel-wise Soft Mask} (KSM), which learns a kernel-wise hybridbinary and real-value soft mask for each task, while using the same backbonemodel. Such a soft mask can be viewed as a superposition of a binary mask and aproperly scaled real-value tensor, which offers a richer representationcapability without low-level kernel support to meet the objective of lowhardware overhead. We validate KSM on multiple benchmark datasets againstrecent state-of-the-art methods (e.g. Piggyback, Packnet, CPG, etc.), whichshows good improvement in both accuracy and training cost.

KSM: Fast Multiple Task Adaption via Kernel-wise Soft Mask Learning