English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In some settings neural networks exhibit a phenomenon known as grokking,where they achieve perfect or near-perfect accuracy on the validation set longafter the same performance has been achieved on the training set. In thispaper, we discover that grokking is not limited to neural networks but occursin other settings such as Gaussian process (GP) classification, GP regressionand linear regression. We also uncover a mechanism by which to induce grokkingon algorithmic datasets via the addition of dimensions containing spuriousinformation. The presence of the phenomenon in non-neural architecturesprovides evidence that grokking is not specific to SGD or weight normregularisation. Instead, grokking may be possible in any setting where solutionsearch is guided by complexity and error. Based on this insight and furthertrends we see in the training trajectories of a Bayesian neural network (BNN)and GP regression model, we make progress towards a more general theory ofgrokking. Specifically, we hypothesise that the phenomenon is governed by theaccessibility of certain regions in the error and complexity landscapes.

Grokking Beyond Neural Networks: An Empirical Exploration with Model Complexity