English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Nowadays, one practical limitation of deep neural network (DNN) is its highdegree of specialization to a single task or domain (e.g., one visual domain).It motivates researchers to develop algorithms that can adapt DNN model tomultiple domains sequentially, while still performing well on the past domains,which is known as multi-domain learning. Almost all conventional methods onlyfocus on improving accuracy with minimal parameter update, while ignoring highcomputing and memory cost during training, which makes it difficult to deploymulti-domain learning into more and more widely used resource-limited edgedevices, like mobile phones, IoT, embedded systems, etc. We observe that largememory used for activation storage is the bottleneck that largely limits thetraining time and cost on edge devices. To reduce training memory usage, whilekeeping the domain adaption accuracy performance, we propose Dynamic AdditiveAttention Adaption ($DA^3$), a novel memory-efficient on-device multi-domainlearning method. $DA^3$ learns a novel additive attention adaptor module, whilefreezing the weights of the pre-trained backbone model for each domain.Differentiating from prior works, such module not only mitigates activationmemory buffering for reducing memory usage during training but also serves as adynamic gating mechanism to reduce the computation cost for fast inference. Wevalidate $DA^3$ on multiple datasets against state-of-the-art methods, whichshows great improvement in both accuracy and training time. Moreover, wedeployed $DA^3$ into the popular NIVDIA Jetson Nano edge GPU, where themeasured experimental results show our proposed $DA^3$ reduces the on-devicetraining memory consumption by 19-37x, and training time by 2x, in comparisonto the baseline methods (e.g., standard fine-tuning, Parallel and Series Res.adaptor, and Piggyback).

$DA^3$:Dynamic Additive Attention Adaption for Memory-EfficientOn-Device Multi-Domain Learning