Open AccessDeep Neural Network Optimization Based on Binary Method for Handling Multi-class Problems
Author(s) -
Yuqi Liu,
Sibo Yang,
Yuan Bao
Publication year - 2024
Publication title -
ieee access
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.587
H-Index - 127
ISSN - 2169-3536
DOI - 10.1109/access.2024.3382195
Subject(s) - aerospace , bioengineering , communication, networking and broadcast technologies , components, circuits, devices and systems , computing and processing , engineered materials, dielectrics and plasmas , engineering profession , fields, waves and electromagnetics , general topics for engineers , geoscience , nuclear engineering , photonics and electrooptics , power, energy and industry applications , robotics and control systems , signal processing and analysis , transportation
In this paper, we conceive a new kind of output layer design in deep neural networks for the multi-class problems. The traditional output layer is set by the one-to-one method. For the one-to-one method, the output layer neuron number is the same as the class number. And the ideal output for the j -th class sample is e j , where e j is j -th unit vector. However, one-to-one method requires too many output neurons, which will increase the number of weights connecting the last-hidden and the output layers. Furthermore, during the process of network training, computation time and cost will greatly increase. We design the binary method for the output layer: Let the class number be k ( k ≥ 3), and 2 a -1 < k ≤ 2 a ( a = ⌈ log 2 k ⌉), then the output layer neuron number is a and the ideal output is designed by binary method. Obviously, the binary method uses less output nodes than the traditional one-to-one method. On this foundation, the number of hidden-output weights will also decrease. On the other hand, while training the deep neural network, the learning efficiency will also be significantly improved. Numerical experiments show that binary method has better classification performance and calculation speed than one-to-one method on the datasets.