English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

When training early-stage deep neural networks (DNNs), generatingintermediate features via convolution or linear layers occupied most of theexecution time. Accordingly, extensive research has been done to reduce thecomputational burden of the convolution or linear layers. In recentmobile-friendly DNNs, however, the relative number of operations involved inprocessing these layers has significantly reduced. As a result, the proportionof the execution time of other layers, such as batch normalization layers, hasincreased. Thus, in this work, we conduct a detailed analysis of the batchnormalization layer to efficiently reduce the runtime overhead in the batchnormalization process. Backed up by the thorough analysis, we present anextremely efficient batch normalization, named LightNorm, and its associatedhardware module. In more detail, we fuse three approximation techniques thatare i) low bit-precision, ii) range batch normalization, and iii) blockfloating point. All these approximate techniques are carefully utilized notonly to maintain the statistics of intermediate feature maps, but also tominimize the off-chip memory accesses. By using the proposed LightNormhardware, we can achieve significant area and energy savings during the DNNtraining without hurting the training accuracy. This makes the proposedhardware a great candidate for the on-device training.

LightNorm: Area and Energy-Efficient Batch Normalization Hardware for On-Device DNN Training