English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep neural networks (DNNs) have shown to provide superb performance in manyreal life applications, but their large computation cost and storagerequirement have prevented them from being deployed to many edge andinternet-of-things (IoT) devices. Sparse deep neural networks, whose majorityweight parameters are zeros, can substantially reduce the computationcomplexity and memory consumption of the models. In real-use scenarios, devicesmay suffer from large fluctuations of the available computation and memoryresources under different environment, and the quality of service (QoS) isdifficult to maintain due to the long tail inferences with large latency.Facing the real-life challenges, we propose to train a sparse model thatsupports multiple sparse levels. That is, a hierarchical structure of weightsare satisfied such that the locations and the values of the non-zero parametersof the more-sparse sub-model area subset of the less-sparse sub-model. In thisway, one can dynamically select the appropriate sparsity level duringinference, while the storage cost is capped by the least sparse sub-model. Wehave verified our methodologies on a variety of DNN models and tasks, includingthe ResNet-50, PointNet++, GNMT, and graph attention networks. We obtain sparsesub-models with an average of 13.38% weights and 14.97% FLOPs, while theaccuracies are as good as their dense counterparts. More-sparse sub-models with5.38% weights and 4.47% of FLOPs, which are subsets of the less-sparse ones,can be obtained with only 3.25% relative accuracy loss.

Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting