Pruning Neural Networks Using Multi-Armed Bandits

The successful application of deep learning has led to increasing expectations of their use in embedded systems. This in turn, has created the need to find ways of reducing the size of neural networks. Decreasing the size of a neural network requires deciding which weights should be removed without compromising accuracy, which is analogous to the kind of problems addressed by multi-arm bandits. Hence, this paper explores the use of multi-armed bandits for reducing the number of parameters of a neural network. Different multi-armed bandit algorithms, namely e-greedy, win-stay, lose-shift, UCB1, KL-UCB, BayesUCB, UGapEb, Successive Rejects and Thompson sampling are evaluated and their performance compared to existing approaches. The results show that multi- armed bandit pruning methods, especially those based on UCB, outperform other pruning methods.