English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reinforcement learning agents need a reward signal to learn successfulpolicies. When this signal is sparse or the corresponding gradient isdeceptive, such agents need a dedicated mechanism to efficiently explore theirsearch space without relying on the reward. Looking for a large diversity ofbehaviors or using Motion Planning (MP) algorithms are two options in thiscontext. In this paper, we build on the common roots between these two optionsto investigate the properties of two diversity search algorithms, the NoveltySearch and the Goal Exploration Process algorithms. These algorithms look fordiversity in an outcome space or behavioral space which is generallyhand-designed to represent what matters for a given task. The relation to MPalgorithms reveals that the smoothness, or lack of smoothness of the mappingbetween the policy parameter space and the outcome space plays a key role inthe search efficiency. In particular, we show empirically that, if the mappingis smooth enough, i.e. if two close policies in the parameter space lead tosimilar outcomes, then diversity algorithms tend to inherit explorationproperties of MP algorithms. By contrast, if it is not, diversity algorithmslose these properties and their performance strongly depends on specificheuristics, notably filtering mechanisms that discard some of the exploredpolicies.

Selection-Expansion: A Unifying Framework for Motion-Planning and Diversity Search Algorithms