Open AccessImplementation of Dropout Neuronal Units Based on Stochastic Memristive Devices in Neural Networks with High Classification Accuracy
Author(s) -
Huang HeMing,
Xiao Yu,
Yang Rui,
Yu YeTian,
He HuiKai,
Wang Zhe,
Guo Xin
Publication year - 2020
Publication title -
advanced science
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 5.388
H-Index - 100
ISSN - 2198-3844
DOI - 10.1002/advs.202001842
Subject(s) - overfitting , dropout (neural networks) , artificial neural network , limit (mathematics) , computer science , artificial intelligence , nonlinear system , materials science , machine learning , mathematics , physics , mathematical analysis , quantum mechanics
Abstract Neural networks based on memristive devices have achieved great progress recently. However, memristive synapses with nonlinearity and asymmetry seriously limit the classification accuracy. Moreover, insufficient number of training samples in many cases also have negative effect on the classification accuracy of neural networks due to overfitting. In this work, dropout neuronal units are developed based on stochastic volatile memristive devices of Ag/Ta 2 O 5 :Ag/Pt. The memristive neural network using the dropout neuronal units effectively solves the problem of overfitting and mitigates the negative effects of the nonideality of memristive synapses, eventually achieves a classification accuracy comparable to the theoretical limit. The stochastic and volatile switching performances of the Ag/Ta 2 O 5 :Ag/Pt device are attributed to the stochastical rupture of the Ag filament under high electrical stress in the Ta 2 O 5 layer, according to the TEM observation and the kinetic Monte Carlo simulation.