English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Due to diverse architectures in deep neural networks (DNNs) with severeoverparameterization, regularization techniques are critical for findingoptimal solutions in the huge hypothesis space. In this paper, we propose aneffective regularization technique, called Neighborhood Region Smoothing (NRS).NRS leverages the finding that models would benefit from converging to flatminima, and tries to regularize the neighborhood region in weight space toyield approximate outputs. Specifically, gap between outputs of models in theneighborhood region is gauged by a defined metric based on Kullback-Leiblerdivergence. This metric provides similar insights with the minimum descriptionlength principle on interpreting flat minima. By minimizing both thisdivergence and empirical loss, NRS could explicitly drive the optimizer towardsconverging to flat minima. We confirm the effectiveness of NRS by performingimage classification tasks across a wide range of model architectures oncommonly-used datasets such as CIFAR and ImageNet, where generalization abilitycould be universally improved. Also, we empirically show that the minima foundby NRS would have relatively smaller Hessian eigenvalues compared to theconventional method, which is considered as the evidence of flat minima.

Neighborhood Region Smoothing Regularization for Finding Flat Minima In Deep Neural Networks