Premium
Weight Programming in DNN Analog Hardware Accelerators in the Presence of NVM Variability
Author(s) -
Mackin Charles,
Tsai Hsinyu,
Ambrogio Stefano,
Narayanan Pritish,
Chen An,
Burr Geoffrey W.
Publication year - 2019
Publication title -
advanced electronic materials
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.25
H-Index - 56
ISSN - 2199-160X
DOI - 10.1002/aelm.201900026
Subject(s) - computer science , perceptron , context (archaeology) , non volatile memory , inference , dimension (graph theory) , crossbar switch , artificial neural network , parallel computing , computer hardware , computer engineering , artificial intelligence , mathematics , paleontology , telecommunications , pure mathematics , biology
Abstract Crossbar arrays of nonvolatile memory (NVM) can potentially accelerate development of deep neural networks (DNNs) by implementing crucial multiply–accumulate (MAC) operations at the location of data. Effective weight‐programming procedures can both minimize the performance impact during training and reduce the down time for inference, where new parameter sets may need to be loaded. Simultaneous weight programming along an entire dimension (e.g., row or column) of a crossbar array in the context of forward inference and training is shown to be important. A framework for determining the optimal hardware conditions in which to program weights is provided, and its efficacy in the presence of considerable NVM variability is explored through simulations. This strategy is shown capable of programming 98–99% of weights effectively, in a manner that is both largely independent of the target weight distribution and highly tolerant to variability in NVM conductance‐versus‐pulse characteristics. The probability that a weight fails to reach its target value, termed P fail , is quantified and the fundamental trade‐off between P fail and weight programming speed is explored. Lastly, the impact of imperfectly programmed weights on DNN test accuracies is examined for various networks, including multi‐layer perceptrons (MLPs) and long short‐term memory (LSTM).