Open AccessPlateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance
Author(s) -
Mark Ainsworth,
Yeonjong Shin
Publication year - 2021
Publication title -
siam journal on scientific computing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.674
H-Index - 147
eISSN - 1095-7197
pISSN - 1064-8275
DOI - 10.1137/20m1353010
Subject(s) - mathematics , plateau (mathematics) , training (meteorology) , gradient descent , descent (aeronautics) , phenomenon , algorithm , artificial neural network , artificial intelligence , mathematical analysis , computer science , physics , quantum mechanics , meteorology
Accelerating Research
Robert Robinson Avenue,
Oxford Science Park, Oxford
OX4 4GP, United Kingdom
Address
John Eccles HouseRobert Robinson Avenue,
Oxford Science Park, Oxford
OX4 4GP, United Kingdom