Open AccessSparseConnect: regularising CNNs on fully connected layers
Author(s) -
Xu Qi,
Pan Gang
Publication year - 2017
Publication title -
electronics letters
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.375
H-Index - 146
ISSN - 1350-911X
DOI - 10.1049/el.2017.2621
Subject(s) - mnist database , convolutional neural network , benchmark (surveying) , computer science , artificial intelligence , pattern recognition (psychology) , training set , deep learning , machine learning , algorithm , geodesy , geography
Deep convolutional neural networks (CNNs) have achieved unprecedented success in many domains. The numerous parameters allow CNNs to learn complex features, but also tend to hinder generalisation by over‐fitting training data. Despite many previously proposed regularisation methods, over‐fitting is one of many problems in training a robust CNN. Among many factors that may lead to over‐fitting, the numerous parameters of fully connected layers (FCLs) of a typical CNN are a contributor to the over‐fitting problem. The authors propose SparseConnect, which alleviates over‐fitting by sparsifying connections to FCLs. Experimental results on three benchmark datasets MNIST and CIFAR10 show SparseConnect outperforms several state‐of‐the‐art regularisation methods.