Improve Discontinuous Output Change in SpikeProp
Author(s) -
Haruhiko Takase,
Hiroharu Kawanaka,
Shinji Tsuruoka
Publication year - 2014
Publication title -
journal of advanced computational intelligence and intelligent informatics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.172
H-Index - 20
eISSN - 1343-0130
pISSN - 1883-8014
DOI - 10.20965/jaciii.2014.p0361
Subject(s) - computer science , weight change , square (algebra) , algorithm , mathematical optimization , mathematics , geometry , weight loss , medicine , obesity
We try to improve discontinuous output change in SpikeProp. The problem is that small variants in input cause significant change in output. We first show that peaks in activity cause the problem. We then propose reducing the height of peaks by weight decay. Through experiments, we conclude that the square type of weight decay is suitable for solving the problem, because it reduces to more than half the patterns that cause discontinuous output change around them.
Accelerating Research
Robert Robinson Avenue,
Oxford Science Park, Oxford
OX4 4GP, United Kingdom
Address
John Eccles HouseRobert Robinson Avenue,
Oxford Science Park, Oxford
OX4 4GP, United Kingdom