English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep neural networks (DNNs) have surpassed human-level accuracy in a varietyof cognitive tasks but at the cost of significant memory/time requirements inDNN training. This limits their deployment in energy and memory limitedapplications that require real-time learning. Matrix-vector multiplications(MVM) and vector-vector outer product (VVOP) are the two most expensiveoperations associated with the training of DNNs. Strategies to improve theefficiency of MVM computation in hardware have been demonstrated with minimalimpact on training accuracy. However, the VVOP computation remains a relativelyless explored bottleneck even with the aforementioned strategies. Stochasticcomputing (SC) has been proposed to improve the efficiency of VVOP computationbut on relatively shallow networks with bounded activation functions andfloating-point (FP) scaling of activation gradients. In this paper, we proposeESSOP, an efficient and scalable stochastic outer product architecture based onthe SC paradigm. We introduce efficient techniques to generalize SC for weightupdate computation in DNNs with the unbounded activation functions (e.g.,ReLU), required by many state-of-the-art networks. Our architecture reduces thecomputational cost by re-using random numbers and replacing certain FPmultiplication operations by bit shift scaling. We show that the ResNet-32network with 33 convolution layers and a fully-connected layer can be trainedwith ESSOP on the CIFAR-10 dataset to achieve baseline comparable accuracy.Hardware design of ESSOP at 14nm technology node shows that, compared to ahighly pipelined FP16 multiplier design, ESSOP is 82.2% and 93.7% better inenergy and area efficiency respectively for outer product computation.

ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning