English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Representing deep neural networks (DNNs) in low-precision is a promisingapproach to enable efficient acceleration and memory reduction. Previousmethods that train DNNs in low-precision typically keep a copy of weights inhigh-precision during the weight updates. Directly training with low-precisionweights leads to accuracy degradation due to complex interactions between thelow-precision number systems and the learning algorithms. To address thisissue, we develop a co-designed low-precision training framework, termedLNS-Madam, in which we jointly design a logarithmic number system (LNS) and amultiplicative weight update algorithm (Madam). We prove that LNS-Madam resultsin low quantization error during weight updates, leading to stable performanceeven if the precision is limited. We further propose a hardware design ofLNS-Madam that resolves practical challenges in implementing an efficientdatapath for LNS computations. Our implementation effectively reduces energyoverhead incurred by LNS-to-integer conversion and partial sum accumulation.Experimental results show that LNS-Madam achieves comparable accuracy tofull-precision counterparts with only 8 bits on popular computer vision andnatural language tasks. Compared to FP32 and FP8, LNS-Madam reduces the energyconsumption by over 90% and 55%, respectively.

LNS-Madam: Low-Precision Training in Logarithmic Number System using Multiplicative Weight Update