English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider feature selection for applications in machine learning where thedimensionality of the data is so large that it exceeds the working memory ofthe (local) computing machine. Unfortunately, current large-scale sketchingalgorithms show poor memory-accuracy trade-off due to the irreversiblecollision and accumulation of the stochastic gradient noise in the sketcheddomain. Here, we develop a second-order ultra-high dimensional featureselection algorithm, called BEAR, which avoids the extra collisions by storingthe second-order gradients in the celebrated Broyden-Fletcher-Goldfarb-Shannon(BFGS) algorithm in Count Sketch, a sublinear memory data structure from thestreaming literature. Experiments on real-world data sets demonstrate that BEARrequires up to three orders of magnitude less memory space to achieve the sameclassification accuracy compared to the first-order sketching algorithms.Theoretical analysis proves convergence of BEAR with rate O(1/t) in titerations of the sketched algorithm. Our algorithm reveals an unexploredadvantage of second-order optimization for memory-constrained sketching ofmodels trained on ultra-high dimensional data sets.

BEAR: Sketching BFGS Algorithm for Ultra-High Dimensional Feature Selection in Sublinear Memory