English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We investigate the problem of corruption robustness in offline reinforcementlearning (RL) with general function approximation, where an adversary cancorrupt each sample in the offline dataset, and the corruption level$\zeta\geq0$ quantifies the cumulative corruption amount over $n$ episodes and$H$ steps. Our goal is to find a policy that is robust to such corruption andminimizes the suboptimality gap with respect to the optimal policy for theuncorrupted Markov decision processes (MDPs). Drawing inspiration from theuncertainty-weighting technique from the robust online RL setting\citep{he2022nearly,ye2022corruptionrobust}, we design a new uncertainty weightiteration procedure to efficiently compute on batched samples and propose acorruption-robust algorithm for offline RL. Notably, under the assumption ofsingle policy coverage and the knowledge of $\zeta$, our proposed algorithmachieves a suboptimality bound that is worsened by an additive factor of$\mathcal O(\zeta \cdot (\text{CC}(\lambda,\hat{\mathcal F},\mathcalZ_n^H))^{1/2} (C(\hat{\mathcal F},\mu))^{-1/2} n^{-1})$ due to the corruption.Here $\text{CC}(\lambda,\hat{\mathcal F},\mathcal Z_n^H)$ is the coveragecoefficient that depends on the regularization parameter $\lambda$, theconfidence set $\hat{\mathcal F}$, and the dataset $\mathcal Z_n^H$, and$C(\hat{\mathcal F},\mu)$ is a coefficient that depends on $\hat{\mathcal F}$and the underlying data distribution $\mu$. When specialized to linear MDPs,the corruption-dependent error term reduces to $\mathcal O(\zeta d n^{-1})$with $d$ being the dimension of the feature map, which matches the existinglower bound for corrupted linear MDPs. This suggests that our analysis is tightin terms of the corruption-dependent term.

Corruption-Robust Offline Reinforcement Learning with General Function Approximation