English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently, a new trend of exploring sparsity for accelerating neural networktraining has emerged, embracing the paradigm of training on the edge. Thispaper proposes a novel Memory-Economic Sparse Training (MEST) frameworktargeting for accurate and fast execution on edge devices. The proposed MESTframework consists of enhancements by Elastic Mutation (EM) and Soft MemoryBound (&S) that ensure superior accuracy at high sparsity ratios. Differentfrom the existing works for sparse training, this current work reveals theimportance of sparsity schemes on the performance of sparse training in termsof accuracy as well as training speed on real edge devices. On top of that, thepaper proposes to employ data efficiency for further acceleration of sparsetraining. Our results suggest that unforgettable examples can be identifiedin-situ even during the dynamic exploration of sparsity masks in the sparsetraining process, and therefore can be removed for further training speedup onedge devices. Comparing with state-of-the-art (SOTA) works on accuracy, ourMEST increases Top-1 accuracy significantly on ImageNet when using the sameunstructured sparsity scheme. Systematical evaluation on accuracy, trainingspeed, and memory footprint are conducted, where the proposed MEST frameworkconsistently outperforms representative SOTA works. A reviewer strongly againstour work based on his false assumptions and misunderstandings. On top of theprevious submission, we employ data efficiency for further acceleration ofsparse training. And we explore the impact of model sparsity, sparsity schemes,and sparse training algorithms on the number of removable training examples.Our codes are publicly available at: https://github.com/boone891214/MEST.

MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge