English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this work, we study the low-rank MDPs with adversarially changed losses inthe full-information feedback setting. In particular, the unknown transitionprobability kernel admits a low-rank matrix decomposition \citep{REPUCB22}, andthe loss functions may change adversarially but are revealed to the learner atthe end of each episode. We propose a policy optimization-based algorithm POLO,and we prove that it attains the$\widetilde{O}(K^{\frac{5}{6}}A^{\frac{1}{2}}d\ln(1+M)/(1-\gamma)^2)$ regretguarantee, where $d$ is rank of the transition kernel (and hence the dimensionof the unknown representations), $A$ is the cardinality of the action space,$M$ is the cardinality of the model class, and $\gamma$ is the discountedfactor. Notably, our algorithm is oracle-efficient and has a regret guaranteewith no dependence on the size of potentially arbitrarily large state space.Furthermore, we also prove an $\Omega(\frac{\gamma^2}{1-\gamma} \sqrt{d A K})$regret lower bound for this problem, showing that low-rank MDPs arestatistically more difficult to learn than linear MDPs in the regretminimization setting. To the best of our knowledge, we present the firstalgorithm that interleaves representation learning, exploration, andexploitation to achieve the sublinear regret guarantee for RL with nonlinearfunction approximation and adversarial losses.

Learning Adversarial Low-rank Markov Decision Processes with Unknown Transition and Full-information Feedback