English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

For very large datasets, random projections (RP) have become the tool ofchoice for dimensionality reduction. This is due to the computationalcomplexity of principal component analysis. However, the recent development ofrandomized principal component analysis (RPCA) has opened up the possibility ofobtaining approximate principal components on very large datasets. In thispaper, we compare the performance of RPCA and RP in dimensionality reductionfor supervised learning. In Experiment 1, study a malware classification taskon a dataset with over 10 million samples, almost 100,000 features, and over 25billion non-zero values, with the goal of reducing the dimensionality to acompressed representation of 5,000 features. In order to apply RPCA to thisdataset, we develop a new algorithm called large sample RPCA (LS-RPCA), whichextends the RPCA algorithm to work on datasets with arbitrarily many samples.We find that classification performance is much higher when using LS-RPCA fordimensionality reduction than when using random projections. In particular,across a range of target dimensionalities, we find that using LS-RPCA reducesclassification error by between 37% and 54%. Experiment 2 generalizes thephenomenon to multiple datasets, feature representations, and classifiers.These findings have implications for a large number of research projects inwhich random projections were used as a preprocessing step for dimensionalityreduction. As long as accuracy is at a premium and the target dimensionality issufficiently less than the numeric rank of the dataset, randomized PCA may be asuperior choice. Moreover, if the dataset has a large number of samples, thenLS-RPCA will provide a method for obtaining the approximate principalcomponents.

Projecting "better than randomly": How to reduce the dimensionality of very large datasets in a way that outperforms random projections