English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The history of learning for control has been an exciting back and forthbetween two broad classes of algorithms: planning and reinforcement learning.Planning algorithms effectively reason over long horizons, but assume access toa local policy and distance metric over collision-free paths. Reinforcementlearning excels at learning policies and the relative values of states, butfails to plan over long horizons. Despite the successes of each method invarious domains, tasks that require reasoning over long horizons with limitedfeedback and high-dimensional observations remain exceedingly challenging forboth planning and reinforcement learning algorithms. Frustratingly, these sortsof tasks are potentially the most useful, as they are simple to design (a humanonly need to provide an example goal state) and avoid reward shaping, which canbias the agent towards finding a sub-optimal solution. We introduce a generalcontrol algorithm that combines the strengths of planning and reinforcementlearning to effectively solve these tasks. Our aim is to decompose the task ofreaching a distant goal state into a sequence of easier tasks, each of whichcorresponds to reaching a subgoal. Planning algorithms can automatically findthese waypoints, but only if provided with suitable abstractions of theenvironment -- namely, a graph consisting of nodes and edges. Our main insightis that this graph can be constructed via reinforcement learning, where agoal-conditioned value function provides edge weights, and nodes are taken tobe previously seen observations in a replay buffer. Using graph search over ourreplay buffer, we can automatically generate this sequence of subgoals, even inimage-based environments. Our algorithm, search on the replay buffer (SoRB),enables agents to solve sparse reward tasks over one hundred steps, andgeneralizes substantially better than standard RL algorithms.

Search on the Replay Buffer: Bridging Planning and Reinforcement Learning