English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As an important framework for safe Reinforcement Learning, the ConstrainedMarkov Decision Process (CMDP) has been extensively studied in the recentliterature. However, despite the rich results under various on-policy learningsettings, there still lacks some essential understanding of the offline CMDPproblems, in terms of both the algorithm design and the information theoreticsample complexity lower bound. In this paper, we focus on solving the CMDPproblems where only offline data are available. By adopting the concept of thesingle-policy concentrability coefficient $C^*$, we establish an$\Omega\left(\frac{\min\left\{|\mathcal{S}||\mathcal{A}|,|\mathcal{S}|+I\right\}C^*}{(1-\gamma)^3\epsilon^2}\right)$ sample complexity lower bound for theoffline CMDP problem, where $I$ stands for the number of constraints. Byintroducing a simple but novel deviation control mechanism, we propose anear-optimal primal-dual learning algorithm called DPDL. This algorithmprovably guarantees zero constraint violation and its sample complexity matchesthe above lower bound except for an $\tilde{\mathcal{O}}((1-\gamma)^{-1})$factor. Comprehensive discussion on how to deal with the unknown constant $C^*$and the potential asynchronous structure on the offline dataset are alsoincluded.

A Near-Optimal Primal-Dual Method for Off-Policy Learning in CMDP