English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Behavioural cloning (BC) is a commonly used imitation learning method toinfer a sequential decision-making policy from expert demonstrations. However,when the quality of the data is not optimal, the resulting behavioural policyalso performs sub-optimally once deployed. Recently, there has been a surge inoffline reinforcement learning methods that hold the promise to extracthigh-quality policies from sub-optimal historical data. A common approach is toperform regularisation during training, encouraging updates during policyevaluation and/or policy improvement to stay close to the underlying data. Inthis work, we investigate whether an offline approach to improving the qualityof the existing data can lead to improved behavioural policies without anychanges in the BC algorithm. The proposed data improvement approach -Trajectory Stitching (TS) - generates new trajectories (sequences of states andactions) by `stitching' pairs of states that were disconnected in the originaldata and generating their connecting new action. By construction, these newtransitions are guaranteed to be highly plausible according to probabilisticmodels of the environment, and to improve a state-value function. Wedemonstrate that the iterative process of replacing old trajectories with newones incrementally improves the underlying behavioural policy. Extensiveexperimental results show that significant performance gains can be achievedusing TS over BC policies extracted from the original data. Furthermore, usingthe D4RL benchmarking suite, we demonstrate that state-of-the-art results areobtained by combining TS with two existing offline learning methodologiesreliant on BC, model-based offline planning (MBOP) and policy constraint(TD3+BC).

Model-based trajectory stitching for improved behavioural cloning and its applications