English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Regularization and normalization have become indispensable components intraining deep neural networks, resulting in faster training and improvedgeneralization performance. We propose the projected error functionregularization loss (PER) that encourages activations to follow the standardnormal distribution. PER randomly projects activations onto one-dimensionalspace and computes the regularization loss in the projected space. PER issimilar to the Pseudo-Huber loss in the projected space, thus taking advantageof both $L^1$ and $L^2$ regularization losses. Besides, PER can capture theinteraction between hidden units by projection vector drawn from a unit sphere.By doing so, PER minimizes the upper bound of the Wasserstein distance of orderone between an empirical distribution of activations and the standard normaldistribution. To the best of the authors' knowledge, this is the first work toregularize activations via distribution matching in the probabilitydistribution space. We evaluate the proposed method on the image classificationtask and the word-level language modeling task.

Regularizing activations in neural networks via distribution matching with the Wasserstein metric