English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Low precision deep neural network (DNN) training is one of the most effectivetechniques for boosting DNNs' training efficiency, as it trims down thetraining cost from the finest bit level. While existing works mostly fix themodel precision during the whole training process, a few pioneering works haveshown that dynamic precision schedules help DNNs converge to a better accuracywhile leading to a lower training cost than their static precision trainingcounterparts. However, existing dynamic low precision training methods rely onmanually designed precision schedules to achieve advantageous efficiency andaccuracy trade-offs, limiting their more comprehensive practical applicationsand achievable performance. To this end, we propose LDP, a Learnable DynamicPrecision DNN training framework that can automatically learn a temporally andspatially dynamic precision schedule during training towards optimal accuracyand efficiency trade-offs. It is worth noting that LDP-trained DNNs are bynature efficient during inference. Furthermore, we visualize the resultingtemporal and spatial precision schedule and distribution of LDP trained DNNs ondifferent tasks to better understand the corresponding DNNs' characteristics atdifferent training stages and DNN layers both during and after training,drawing insights for promoting further innovations. Extensive experiments andablation studies (seven networks, five datasets, and three tasks) show that theproposed LDP consistently outperforms state-of-the-art (SOTA) low precision DNNtraining techniques in terms of training efficiency and achieved accuracytrade-offs. For example, in addition to having the advantage of beingautomated, our LDP achieves a 0.31\% higher accuracy with a 39.1\% lowercomputational cost when training ResNet-20 on CIFAR-10 as compared with thebest SOTA method.

LDP: Learnable Dynamic Precision for Efficient Deep Neural Network Training and Inference