English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many real-world applications, such as those in medical domains,recommendation systems, etc, can be formulated as large state spacereinforcement learning problems with only a small budget of the number ofpolicy changes, i.e., low switching cost. This paper focuses on the linearMarkov Decision Process (MDP) recently studied in [Yang et al 2019, Jin et al2020] where the linear function approximation is used for generalization on thelarge state space. We present the first algorithm for linear MDP with a lowswitching cost. Our algorithm achieves an$\widetilde{O}\left(\sqrt{d^3H^4K}\right)$ regret bound with a near-optimal$O\left(d H\log K\right)$ global switching cost where $d$ is the featuredimension, $H$ is the planning horizon and $K$ is the number of episodes theagent plays. Our regret bound matches the best existing polynomial algorithm by[Jin et al 2020] and our switching cost is exponentially smaller than theirs.When specialized to tabular MDP, our switching cost bound improves those in[Bai et al 2019, Zhang et al 20020]. We complement our positive result with an$\Omega\left(dH/\log d\right)$ global switching cost lower bound for anyno-regret algorithm.

A Provably Efficient Algorithm for Linear Markov Decision Process with Low Switching Cost