English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In Part I, we defined a LASSO condition number and developed an algorithm --for computing support sets (feature selection) of the LASSO minimisationproblem -- that runs in polynomial time in the number of variables and thelogarithm of the condition number. The algorithm is trustworthy in the sensethat if the condition number is infinite, the algorithm will run forever andnever produce an incorrect output. In this Part II article, we demonstrate howfinite precision algorithms (for example algorithms running floating pointarithmetic) will fail on open sets when the condition number is large -- butstill finite. This augments Part I's result: If an algorithm takes inputs froman open set that includes at least one point with an infinite condition number,it fails to compute the correct support set for all inputs within that set.Hence, for any finite precision algorithm working on open sets for the LASSOproblem with random inputs, our LASSO condition number -- as a random variable-- will estimate the probability of success/failure of the algorithm. We showthat a finite precision version of our algorithm works on traditional Gaussiandata for LASSO with high probability. The algorithm is trustworthy,specifically, in the random cases where the algorithm fails, it will notproduce an output. Finally, we demonstrate classical random ensembles for whichthe condition number will be large with high probability, and hence where anyfinite precision algorithm on open sets will fail. We show numerically howcommercial software fails on these cases.

When can you trust feature selection? -- II: On the effects of random data on condition in statistics and optimisation