English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently, methods such as Decision Transformer that reduce reinforcementlearning to a prediction task and solve it via supervised learning (RvS) havebecome popular due to their simplicity, robustness to hyperparameters, andstrong overall performance on offline RL tasks. However, simply conditioning aprobabilistic model on a desired return and taking the predicted action canfail dramatically in stochastic environments since trajectories that result ina return may have only achieved that return due to luck. In this work, wedescribe the limitations of RvS approaches in stochastic environments andpropose a solution. Rather than simply conditioning on the return of a singletrajectory as is standard practice, our proposed method, ESPER, learns tocluster trajectories and conditions on average cluster returns, which areindependent from environment stochasticity. Doing so allows ESPER to achievestrong alignment between target return and expected performance in realenvironments. We demonstrate this in several challenging stochastic offline-RLtasks including the challenging puzzle game 2048, and Connect Four playingagainst a stochastic opponent. In all tested domains, ESPER achievessignificantly better alignment between the target return and achieved returnthan simply conditioning on returns. ESPER also achieves higher maximumperformance than even the value-based baselines.

You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments