English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Goal-Conditioned Reinforcement Learning (RL) problems often have access tosparse rewards where the agent receives a reward signal only when it hasachieved the goal, making policy optimization a difficult problem. Severalworks augment this sparse reward with a learned dense reward function, but thiscan lead to sub-optimal policies if the reward is misaligned. Moreover, recentworks have demonstrated that effective shaping rewards for a particular problemcan depend on the underlying learning algorithm. This paper introduces a novelway to encourage exploration called $f$-Policy Gradients, or $f$-PG. $f$-PGminimizes the f-divergence between the agent's state visitation distributionand the goal, which we show can lead to an optimal policy. We derive gradientsfor various f-divergences to optimize this objective. Our learning paradigmprovides dense learning signals for exploration in sparse reward settings. Wefurther introduce an entropy-regularized policy optimization objective, that wecall $state$-MaxEnt RL (or $s$-MaxEnt RL) as a special case of our objective.We show that several metric-based shaping rewards like L2 can be used with$s$-MaxEnt RL, providing a common ground to study such metric-based shapingrewards with efficient exploration. We find that $f$-PG has better performancecompared to standard policy gradient methods on a challenging gridworld as wellas the Point Maze and FetchReach environments. More information on our websitehttps://agarwalsiddhant10.github.io/projects/fpg.html.

$f$-Policy Gradients: A General Framework for Goal Conditioned RL using $f$-Divergences