English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recent years, deeper and wider neural networks have shown excellentperformance in computer vision tasks, while their enormous amount of parametersresults in increased computational cost and overfitting. Several methods havebeen proposed to compress the size of the networks without reducing networkperformance. Network pruning can reduce redundant and unnecessary parametersfrom a network. Knowledge distillation can transfer the knowledge of deeper andwider networks to smaller networks. The performance of the smaller networkobtained by these methods is bounded by the predefined network. Neuralarchitecture search has been proposed, which can search automatically thearchitecture of the networks to break the structure limitation. Also, there isa dynamic configuration method to train networks incrementally as sub-networks.In this paper, we present a novel incremental training algorithm for deepneural networks called planting. Our planting can search the optimal networkarchitecture with smaller number of parameters for improving the networkperformance by augmenting channels incrementally to layers of the initialnetworks while keeping the earlier trained parameters fixed. Also, we proposeusing the knowledge distillation method for training the channels planted. Bytransferring the knowledge of deeper and wider networks, we can grow thenetworks effectively and efficiently. We evaluate the effectiveness of theproposed method on different datasets such as CIFAR-10/100 and STL-10. For theSTL-10 dataset, we show that we are able to achieve comparable performance withonly 7% parameters compared to the larger network and reduce the overfittingcaused by a small amount of the data.

Channel Planting for Deep Neural Networks using Knowledge Distillation