English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Logistic regression is one of the most popular methods in binaryclassification, wherein estimation of model parameters is carried out bysolving the maximum likelihood (ML) optimization problem, and the ML estimatoris defined to be the optimal solution of this problem. It is well known thatthe ML estimator exists when the data is non-separable, but fails to exist whenthe data is separable. First-order methods are the algorithms of choice forsolving large-scale instances of the logistic regression problem. In thispaper, we introduce a pair of condition numbers that measure the degree ofnon-separability or separability of a given dataset in the setting of binaryclassification, and we study how these condition numbers relate to and informthe properties and the convergence guarantees of first-order methods. When thetraining data is non-separable, we show that the degree of non-separabilitynaturally enters the analysis and informs the properties and convergenceguarantees of two standard first-order methods: steepest descent (for any givennorm) and stochastic gradient descent. Expanding on the work of Bach, we alsoshow how the degree of non-separability enters into the analysis of linearconvergence of steepest descent (without needing strong convexity), as well asthe adaptive convergence of stochastic gradient descent. When the training datais separable, first-order methods rather curiously have good empirical success,which is not well understood in theory. In the case of separable data, wedemonstrate how the degree of separability enters into the analysis of $\ell_2$steepest descent and stochastic gradient descent for deliveringapproximate-maximum-margin solutions with associated computational guaranteesas well. This suggests that first-order methods can lead to statisticallymeaningful solutions in the separable case, even though the ML solution doesnot exist.

Condition Number Analysis of Logistic Regression, and its Implications for Standard First-Order Solution Methods