Open AccessPresB-Net: parametric binarized neural network with learnable activations and shuffled grouped convolution
Author(s) -
Jungwoo Shin,
HyunJin Kim
Publication year - 2022
Publication title -
peerj. computer science
Language(s) - English
Resource type - Journals
ISSN - 2376-5992
DOI - 10.7717/peerj-cs.842
Subject(s) - normalization (sociology) , artificial neural network , computer science , parametric statistics , convolution (computer science) , algorithm , artificial intelligence , inference , pattern recognition (psychology) , computation , mathematics , statistics , sociology , anthropology
In this study, we present a novel performance-enhancing binarized neural network model called PresB-Net: Parametric Binarized Neural Network. A binarized neural network (BNN) model can achieve fast output computation with low hardware costs by using binarized weights and features. However, performance degradation is the most critical problem in BNN models. Our PresB-Net combines several state-of-the-art BNN structures including the learnable activation with additional trainable parameters and shuffled grouped convolution. Notably, we propose a new normalization approach, which reduces the imbalance between the shuffled groups occurring in shuffled grouped convolutions. Besides, the proposed normalization approach helps gradient convergence so that the unstableness of the learning can be amortized when applying the learnable activation. Our novel BNN model enhances the classification performance compared with other existing BNN models. Notably, the proposed PresB-Net-18 achieves 73.84% Top-1 inference accuracy for the CIFAR-100 dataset, outperforming other existing counterparts.