English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the realm of efficient on-device learning under extreme memory andcomputation constraints, a significant gap in successful approaches persists.Although considerable effort has been devoted to efficient inference, the mainobstacle to efficient learning is the prohibitive cost of backpropagation. Theresources required to compute gradients and update network parameters oftenexceed the limits of tightly constrained memory budgets. This paper challengesconventional wisdom and proposes a series of experiments that reveal theexistence of superior sub-networks. Furthermore, we hint at the potential forsubstantial gains through a dynamic neuron selection strategy when fine-tuninga target task. Our efforts extend to the adaptation of a recent dynamic neuronselection strategy pioneered by Bragagnolo et al. (NEq), revealing itseffectiveness in the most stringent scenarios. Our experiments demonstrate, inthe average case, the superiority of a NEq-inspired approach over a randomselection. This observation prompts a compelling avenue for further explorationin the area, highlighting the opportunity to design a new class of algorithmsdesigned to facilitate parameter update selection. Our findings usher in a newera of possibilities in the field of on-device learning under extremeconstraints and encourage the pursuit of innovative strategies for efficient,resource-friendly model fine-tuning.

Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint