English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Model-free reinforcement learning (RL) is a powerful approach for learningcontrol policies directly from high-dimensional state and observation. However,it tends to be data-inefficient, which is especially costly in robotic learningtasks. On the other hand, optimal control does not require data if the systemmodel is known, but cannot scale to models with high-dimensional states andobservations. To exploit benefits of both model-free RL and optimal control, wepropose time-to-reach-based (TTR-based) reward shaping, an optimalcontrol-inspired technique to alleviate data inefficiency while retainingadvantages of model-free RL. This is achieved by summarizing key system modelinformation using a TTR function to greatly speed up the RL process, as shownin our simulation results. The TTR function is defined as the minimum timerequired to move from any state to the goal under assumed system dynamicsconstraints. Since the TTR function is computationally intractable for systemswith high-dimensional states, we compute it for approximate, lower-dimensionalsystem models that still captures key dynamic behaviors. Our approach can beflexibly and easily incorporated into any model-free RL algorithm withoutaltering the original algorithm structure, and is compatible with any othertechniques that may facilitate the RL process. We evaluate our approach on tworepresentative robotic learning tasks and three well-known model-free RLalgorithms, and show significant improvements in data efficiency andperformance.

TTR-Based Reward for Reinforcement Learning with Implicit Model Priors