English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While highly expressive parametric models including deep neural networks havean advantage to model complicated concepts, training such highly non-linearmodels is known to yield a high risk of notorious overfitting. To address thisissue, this study considers a $(k,q)$th order variation regularization($(k,q)$-VR), which is defined as the $q$th-powered integral of the absolute$k$th order derivative of the parametric models to be trained; penalizing the$(k,q)$-VR is expected to yield a smoother function, which is expected to avoidoverfitting. Particularly, $(k,q)$-VR encompasses the conventional(general-order) total variation with $q=1$. While the $(k,q)$-VR terms appliedto general parametric models are computationally intractable due to theintegration, this study provides a stochastic optimization algorithm, that canefficiently train general models with the $(k,q)$-VR without conductingexplicit numerical integration. The proposed approach can be applied to thetraining of even deep neural networks whose structure is arbitrary, as it canbe implemented by only a simple stochastic gradient descent algorithm andautomatic differentiation. Our numerical experiments demonstrate that theneural networks trained with the $(k,q)$-VR terms are more ``resilient'' thanthose with the conventional parameter regularization. The proposed algorithmalso can be extended to the physics-informed training of neural networks(PINNs).

A stochastic optimization approach to train non-linear neural networks with a higher-order variation regularization