English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In this paper, we propose and analyze zeroth-order stochastic approximationalgorithms for nonconvex and convex optimization, with a focus on addressingconstrained optimization, high-dimensional setting and saddle-point avoiding.To handle constrained optimization, we first propose generalizations of theconditional gradient algorithm achieving rates similar to the standardstochastic gradient algorithm using only zeroth-order information. Tofacilitate zeroth-order optimization in high-dimensions, we explore theadvantages of structural sparsity assumptions. Specifically, (i) we highlightan implicit regularization phenomenon where the standard stochastic gradientalgorithm with zeroth-order information adapts to the sparsity of the problemat hand by just varying the step-size and (ii) propose a truncated stochasticgradient algorithm with zeroth-order information, whose rate of convergencedepends only poly-logarithmically on the dimensionality. We next focus onavoiding saddle-points in non-convex setting. Towards that, we interpret theGaussian smoothing technique for estimating gradient based on zeroth-orderinformation as an instantiation of first-order Stein's identity. Based on this,we provide a novel linear-(in dimension) time estimator of the Hessian matrixof a function using only zeroth-order information, which is based onsecond-order Stein's identity. We then provide an algorithm for avoidingsaddle-points, which is based on a zeroth-order cubic regularization Newton'smethod and discuss its convergence rates.

Zeroth-order Nonconvex Stochastic Optimization: Handling Constraints, High-Dimensionality and Saddle-Points