English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper studies model-based bandit and reinforcement learning (RL) withnonlinear function approximations. We propose to study convergence toapproximate local maxima because we show that global convergence isstatistically intractable even for one-layer neural net bandit with adeterministic reward. For both nonlinear bandit and RL, the paper presents amodel-based algorithm, Virtual Ascent with Online Model Learner (ViOlin), whichprovably converges to a local maximum with sample complexity that only dependson the sequential Rademacher complexity of the model class. Our results implynovel global or local regret bounds on several concrete settings such as linearbandit with finite or sparse model class, and two-layer neural net bandit. Akey algorithmic insight is that optimism may lead to over-exploration even fortwo-layer neural net model class. On the other hand, for convergence to localmaxima, it suffices to maximize the virtual return if the model can alsoreasonably predict the size of the gradient and Hessian of the real return.

Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature