English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reducing the variance of the gradient estimator is known to improve theconvergence rate of stochastic gradient-based optimization and samplingalgorithms. One way of achieving variance reduction is to design importancesampling strategies. Recently, the problem of designing such schemes wasformulated as an online learning problem with bandit feedback, and algorithmswith sub-linear static regret were designed. In this work, we build on thisframework and propose Avare, a simple and efficient algorithm for adaptiveimportance sampling for finite-sum optimization and sampling with decreasingstep-sizes. Under standard technical conditions, we show that Avare achieves$\mathcal{O}(T^{2/3})$ and $\mathcal{O}(T^{5/6})$ dynamic regret for SGD andSGLD respectively when run with $\mathcal{O}(1/t)$ step sizes. We achieve thisdynamic regret bound by leveraging our knowledge of the dynamics defined by thealgorithm, and combining ideas from online learning and variance-reducedstochastic optimization. We validate empirically the performance of ouralgorithm and identify settings in which it leads to significant improvements.

Adaptive Importance Sampling for Finite-Sum Optimization and Sampling with Decreasing Step-Sizes