English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning has achieved state-of-the-art accuracies on several computervision tasks. However, the computational and energy requirements associatedwith training such deep neural networks can be quite high. In this paper, wepropose a cumulative training strategy with Net2Net transformation thatachieves training computational efficiency without incurring large accuracyloss, in comparison to a model trained from scratch. We achieve this by firsttraining a small network (with lesser parameters) on a small subset of theoriginal dataset, and then gradually expanding the network using Net2Nettransformation to train incrementally on larger subsets of the dataset. Thisincremental training strategy with Net2Net utilizes function-preservingtransformations that transfers knowledge from each previous small network tothe next larger network, thereby, reducing the overall training complexity. Ourexperiments demonstrate that compared with training from scratch, cumulativetraining yields ~2x reduction in computational complexity for trainingTinyImageNet using VGG19 at iso-accuracy. Besides training efficiency, a keyadvantage of our cumulative training strategy is that we can perform pruningduring Net2Net expansion to obtain a final network with optimal configuration(~0.4x lower inference compute complexity) compared to conventional trainingfrom scratch. We also demonstrate that the final network obtained fromcumulative training yields better generalization performance and noiserobustness. Further, we show that mutual inference from all the networkscreated with cumulative Net2Net expansion enables improved adversarial inputdetection.

Energy-efficient and Robust Cumulative Training with Net2Net Transformation