English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Iterative first-order methods such as gradient descent and its variants arewidely used for solving optimization and machine learning problems. There hasbeen recent interest in analytic or numerically efficient methods for computingworst-case performance bounds for such algorithms, for example over the classof strongly convex loss functions. A popular approach is to assume thealgorithm has a fixed size (fixed dimension, or memory) and that its structureis parameterized by one or two hyperparameters, for example a learning rate anda momentum parameter. Then, a Lyapunov function is sought to certify robuststability and subsequent optimization can be performed to find optimalhyperparameter tunings. In the present work, we instead fix the constraintsthat characterize the loss function and apply techniques from robust controlsynthesis to directly search over algorithms. This approach yields strongerresults than those previously available, since the bounds produced hold overalgorithms with an arbitrary, but finite, amount of memory rather than justholding for algorithms with a prescribed structure.

Direct Synthesis of Iterative Algorithms With Bounds on Achievable Worst-Case Convergence Rate