Premium
Modified online Newton step based on elementwise multiplication
Author(s) -
Singh Charanjeet,
Sharma Anuj
Publication year - 2020
Publication title -
computational intelligence
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.353
H-Index - 52
eISSN - 1467-8640
pISSN - 0824-7935
DOI - 10.1111/coin.12298
Subject(s) - hessian matrix , regret , mathematics , multiplication (music) , matrix (chemical analysis) , algorithm , computation , matrix multiplication , dimension (graph theory) , computer science , newton's method , arithmetic , statistics , combinatorics , nonlinear system , materials science , physics , quantum mechanics , composite material , quantum
The second‐order method using a Newton step is a suitable technique in online learning to guarantee a regret bound. The large data are a challenge in the Newton method to store second‐order matrices such as the hessian. In this article, we have proposed a modified online Newton step that stores first‐ and second‐order matrices of dimension m (classes) by d (features). We have used elementwise arithmetic operations to maintain the size of matrices. The modified second‐order matrix size results in faster computations. Also, the mistake rate is on par with respect to popular methods in the literature. The experimental outcome indicates that proposed method could be helpful to handle large multiclass datasets on common desktop machines using second‐order method as the Newton step.