English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the problem of model-free reinforcement learning, which is oftensolved following the principle of Generalized Policy Iteration (GPI). While GPIis typically an interplay between policy evaluation and policy improvement,most conventional model-free methods assume the independence of the granularityand other details of the GPI steps, despite of the inherent connections betweenthem. In this paper, we present a method that regularizes the inconsistencybetween policy evaluation and policy improvement, leading to a conflict averseGPI solution with reduced functional approximation error. To this end, weformulate a novel learning paradigm where taking the policy evaluation step isequivalent to some compensation of performing policy improvement, and thuseffectively alleviates the gradient conflict between the two GPI steps. We alsoshow that the form of our proposed solution is equivalent to performingentropy-regularized policy improvement and therefore prevents the policy frombeing trapped into suboptimal solutions. We conduct extensive experiments toevaluate our method on the Arcade Learning Environment (ALE). Empirical resultsshow that our method outperforms several strong baselines in major evaluationdomains.

CASA: Bridging the Gap between Policy Improvement and Policy Evaluation with Conflict Averse Policy Iteration