English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Monte-Carlo planning, as exemplified by Monte-Carlo Tree Search (MCTS), hasdemonstrated remarkable performance in applications with finite spaces. In thispaper, we consider Monte-Carlo planning in an environment with continuousstate-action spaces, a much less understood problem with important applicationsin control and robotics. We introduce POLY-HOOT, an algorithm that augmentsMCTS with a continuous armed bandit strategy named Hierarchical OptimisticOptimization (HOO) (Bubeck et al., 2011). Specifically, we enhance HOO by usingan appropriate polynomial, rather than logarithmic, bonus term in the upperconfidence bounds. Such a polynomial bonus is motivated by its empiricalsuccesses in AlphaGo Zero (Silver et al., 2017b), as well as its significantrole in achieving theoretical guarantees of finite space MCTS (Shah et al.,2019). We investigate, for the first time, the regret of the enhanced HOOalgorithm in non-stationary bandit problems. Using this result as a buildingblock, we establish non-asymptotic convergence guarantees for POLY-HOOT: thevalue estimate converges to an arbitrarily small neighborhood of the optimalvalue function at a polynomial rate. We further provide experimental resultsthat corroborate our theoretical findings.

POLY-HOOT: Monte-Carlo Planning in Continuous Space MDPs with Non-Asymptotic Analysis