English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Offline reinforcement learning seeks to utilize offline (observational) datato guide the learning of (causal) sequential decision making strategies. Thehope is that offline reinforcement learning coupled with function approximationmethods (to deal with the curse of dimensionality) can provide a means to helpalleviate the excessive sample complexity burden in modern sequential decisionmaking problems. However, the extent to which this broader approach can beeffective is not well understood, where the literature largely consists ofsufficient conditions.  This work focuses on the basic question of what are necessaryrepresentational and distributional conditions that permit provablesample-efficient offline reinforcement learning. Perhaps surprisingly, our mainresult shows that even if: i) we have realizability in that the true valuefunction of \emph{every} policy is linear in a given set of features and 2) ouroff-policy data has good coverage over all features (under a strong spectralcondition), then any algorithm still (information-theoretically) requires anumber of offline samples that is exponential in the problem horizon in orderto non-trivially estimate the value of \emph{any} given policy. Our resultshighlight that sample-efficient offline policy evaluation is simply notpossible unless significantly stronger conditions hold; such conditions includeeither having low distribution shift (where the offline data distribution isclose to the distribution of the policy to be evaluated) or significantlystronger representational conditions (beyond realizability).

What are the Statistical Limits of Offline RL with Linear Function Approximation?