English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

High-dimensional data are ubiquitous in contemporary science and findingmethods to compress them is one of the primary goals of machine learning. Givena dataset lying in a high-dimensional space (in principle hundreds to severalthousands of dimensions), it is often useful to project it onto alower-dimensional manifold, without loss of information. Identifying theminimal dimension of such manifold is a challenging problem known in theliterature as intrinsic dimension estimation (IDE). Traditionally, most IDEalgorithms are either based on multiscale principal component analysis (PCA) oron the notion of correlation dimension (and more in general onk-nearest-neighbors distances). These methods are affected, in different ways,by a severe curse of dimensionality. In particular, none of the existingalgorithms can provide accurate ID estimates in the extreme locallyundersampled regime, i.e. in the limit where the number of samples in any localpatch of the manifold is less than (or of the same order of) the ID of thedataset. Here we introduce a new ID estimator that leverages on simpleproperties of the tangent space of a manifold to overcome these shortcomings.The method is based on the full correlation integral, going beyond the limit ofsmall radius used for the estimation of the correlation dimension. Ourestimator alleviates the extreme undersampling problem, intractable with othermethods. Based on this insight, we explore a multiscale generalization of thealgorithm. We show that it is capable of (i) identifying multipledimensionalities in a dataset, and (ii) providing accurate estimates of the IDof extremely curved manifolds. In particular, we test the method on manifoldsgenerated from global transformations of high-contrast images, relevant forinvariant object recognition and considered a challenge for state-of-the-art IDestimators.

Intrinsic dimension estimation for locally undersampled data