English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose theoretical analyses of a modified natural gradient descent methodin the neural network function space based on the eigendecompositions of neuraltangent kernel and Fisher information matrix. We firstly present analyticalexpression for the function learned by this modified natural gradient under theassumptions of Gaussian distribution and infinite width limit. Thus, weexplicitly derive the generalization error of the learned neural networkfunction using theoretical methods from eigendecomposition and statisticstheory. By decomposing of the total generalization error attributed todifferent eigenspace of the kernel in function space, we propose a criterionfor balancing the errors stemming from training set and the distributiondiscrepancy between the training set and the true data. Through this approach,we establish that modifying the training direction of the neural network infunction space leads to a reduction in the total generalization error.Furthermore, We demonstrate that this theoretical framework is capable toexplain many existing results of generalization enhancing methods. Thesetheoretical results are also illustrated by numerical examples on syntheticdata.

Modify Training Directions in Function Space to Reduce Generalization Error