English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this work, we focus on the study of stochastic zeroth-order (ZO)optimization which does not require first-order gradient information and usesonly function evaluations. The problem of ZO optimization has emerged in manyrecent machine learning applications, where the gradient of the objectivefunction is either unavailable or difficult to compute. In such cases, we canapproximate the full gradients or stochastic gradients through function valuebased gradient estimates. Here, we propose a novel hybrid gradient estimator(HGE), which takes advantage of the query-efficiency of random gradientestimates as well as the variance-reduction of coordinate-wise gradientestimates. We show that with a graceful design in coordinate importancesampling, the proposed HGE-based ZO optimization method is efficient both interms of iteration complexity as well as function query cost. We provide athorough theoretical analysis of the convergence of our proposed method fornon-convex, convex, and strongly-convex optimization. We show that theconvergence rate that we derive generalizes the results for some prominentexisting methods in the nonconvex case, and matches the optimal result in theconvex case. We also corroborate the theory with a real-world black-box attackgeneration application to demonstrate the empirical advantage of our methodover state-of-the-art ZO optimization approaches.

Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework