English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Open

Presents the pdf analysis as premium feature

PDF Analysis

Insights

Presents the summarisation as premium feature

Summarisation

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

We introduce a new regularization method for Artificial Neural Networks(ANNs) based on Kernel Flows (KFs). KFs were introduced as a method for kernelselection in regression/kriging based on the minimization of the loss ofaccuracy incurred by halving the number of interpolation points in randombatches of the dataset. Writing $f_\theta(x) = \big(f^{(n)}_{\theta_n}\circf^{(n-1)}_{\theta_{n-1}} \circ \dots \circ f^{(1)}_{\theta_1}\big)(x)$ for thefunctional representation of compositional structure of the ANN, the innerlayers outputs $h^{(i)}(x) = \big(f^{(i)}_{\theta_i}\circf^{(i-1)}_{\theta_{i-1}} \circ \dots \circ f^{(1)}_{\theta_1}\big)(x)$ define ahierarchy of feature maps and kernels $k^{(i)}(x,x')=\exp(- \gamma_i\|h^{(i)}(x)-h^{(i)}(x')\|_2^2)$. When combined with a batch of the datasetthese kernels produce KF losses $e_2^{(i)}$ (the $L^2$ regression errorincurred by using a random half of the batch to predict the other half)depending on parameters of inner layers $\theta_1,\ldots,\theta_i$ (and$\gamma_i$). The proposed method simply consists in aggregating a subset ofthese KF losses with a classical output loss. We test the proposed method onCNNs and WRNs without alteration of structure nor output classifier and reportreduced test errors, decreased generalization gaps, and increased robustness todistribution shift without significant increase in computational complexity. Wesuspect that these results might be explained by the fact that whileconventional training only employs a linear functional (a generalized moment)of the empirical distribution defined by the dataset and can be prone totrapping in the Neural Tangent Kernel regime (under over-parameterizations),the proposed loss function (defined as a nonlinear functional of the empiricaldistribution) effectively trains the underlying kernel defined by the CNNbeyond regressing the data with that kernel.

Deep regularization and direct training of the inner layers of Neural Networks with Kernel Flows