English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The flexibility of decision boundaries in neural networks that are unguidedby training data is a well-known problem typically resolved with generalizationmethods. A surprising result from recent knowledge distillation (KD) literatureis that random, untrained, and equally structured teacher networks can alsovastly improve generalization performance. It raises the possibility ofexistence of undiscovered assumptions useful for generalization on an uncertainregion. In this paper, we shed light on the assumptions by analyzing decisionboundaries and confidence distributions of both simple and KD-basedgeneralization methods. Assuming that a decision boundary exists to representthe most general tendency of distinction on an input sample space (i.e., thesimplest hypothesis), we show the various limitations of methods when using thehypothesis. To resolve these limitations, we propose matured dumb teacher basedKD, conservatively transferring the hypothesis for generalization of thestudent without massive destruction of trained information. In practicalexperiments on feed-forward and convolution neural networks for imageclassification tasks on MNIST, CIFAR-10, and CIFAR-100 datasets, the proposedmethod shows stable improvement to the best test performance in the grid searchof hyperparameters. The analysis and results imply that the proposed method canprovide finer generalization than existing methods.

Learning from Matured Dumb Teacher for Fine Generalization