English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

We study model evaluation and model selection from the perspective ofgeneralization ability (GA): the ability of a model to predict outcomes in newsamples from the same population. We believe that GA is one way formally toaddress concerns about the external validity of a model. The GA of a modelestimated on a sample can be measured by its empirical out-of-sample errors,called the generalization errors (GE). We derive upper bounds for the GE, whichdepend on sample sizes, model complexity and the distribution of the lossfunction. The upper bounds can be used to evaluate the GA of a model, ex ante.We propose using generalization error minimization (GEM) as a framework formodel selection. Using GEM, we are able to unify a big class of penalizedregression estimators, including lasso, ridge and bridge, under the same set ofassumptions. We establish finite-sample and asymptotic properties (including$\mathcal{L}_2$-consistency) of the GEM estimator for both the $n \geqslant p$and the $n < p$ cases. We also derive the $\mathcal{L}_2$-distance between thepenalized and corresponding unpenalized regression estimates. In practice, GEMcan be implemented by validation or cross-validation. We show that the GEbounds can be used for selecting the optimal number of folds in $K$-foldcross-validation. We propose a variant of $R^2$, the $GR^2$, as a measure ofGA, which considers both both in-sample and out-of-sample goodness of fit.Simulations are used to demonstrate our key results.

Generalization error minimization: a new approach to model evaluation and selection with an application to penalized regression