English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This paper establishes a precise high-dimensional asymptotic theory forboosting on separable data, taking statistical and computational perspectives.We consider a high-dimensional setting where the number of features (weaklearners) $p$ scales with the sample size $n$, in an overparametrized regime.Under a class of statistical models, we provide an exact analysis of thegeneralization error of boosting when the algorithm interpolates the trainingdata and maximizes the empirical $\ell_1$-margin. Further, we explicitly pindown the relation between the boosting test error and the optimal Bayes error,as well as the proportion of active features at interpolation (with zeroinitialization). In turn, these precise characterizations answer certainquestions raised in \cite{breiman1999prediction, schapire1998boosting}surrounding boosting, under assumed data generating processes. At the heart ofour theory lies an in-depth study of the maximum-$\ell_1$-margin, which can beaccurately described by a new system of non-linear equations; to analyze thismargin, we rely on Gaussian comparison techniques and develop a novel uniformdeviation argument. Our statistical and computational arguments can handle (1)any finite-rank spiked covariance model for the feature distribution and (2)variants of boosting corresponding to general $\ell_q$-geometry, $q \in [1,2]$. As a final component, via the Lindeberg principle, we establish auniversality result showcasing that the scaled $\ell_1$-margin (asymptotically)remains the same, whether the covariates used for boosting arise from anon-linear random feature model or an appropriately linearized model withmatching moments.

A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-$\ell_1$-Norm Interpolated Classifiers