English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Double-descent refers to the unexpected drop in test loss of a learningalgorithm beyond an interpolating threshold with over-parameterization, whichis not predicted by information criteria in their classical forms due to thelimitations in the standard asymptotic approach. We update these analyses usingthe information risk minimization framework and provide Akaike InformationCriterion (AIC) and Bayesian Information Criterion (BIC) for models learned bythe Gibbs algorithm. Notably, the penalty terms for the Gibbs-based AIC and BICcorrespond to specific information measures, i.e., symmetrized KL informationand KL divergence. We extend this information-theoretic analysis toover-parameterized models by providing two different Gibbs-based BICs tocompute the marginal likelihood of random feature models in the regime wherethe number of parameters $p$ and the number of samples $n$ tend to infinity,with $p/n$ fixed. Our experiments demonstrate that the Gibbs-based BIC canselect the high-dimensional model and reveal the mismatch between marginallikelihood and population risk in the over-parameterized regime, providing newinsights to understand double-descent.

Gibbs-Based Information Criteria and the Over-Parameterized Regime