English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present GradientDICE for estimating the density ratio between the statedistribution of the target policy and the sampling distribution in off-policyreinforcement learning. GradientDICE fixes several problems of GenDICE (Zhanget al., 2020), the state-of-the-art for estimating such density ratios. Namely,the optimization problem in GenDICE is not a convex-concave saddle-pointproblem once nonlinearity in optimization variable parameterization isintroduced to ensure positivity, so any primal-dual algorithm is not guaranteedto converge or find the desired solution. However, such nonlinearity isessential to ensure the consistency of GenDICE even with a tabularrepresentation. This is a fundamental contradiction, resulting from GenDICE'soriginal formulation of the optimization problem. In GradientDICE, we optimizea different objective from GenDICE by using the Perron-Frobenius theorem andeliminating GenDICE's use of divergence. Consequently, nonlinearity inparameterization is not necessary for GradientDICE, which is provablyconvergent under linear function approximation.

GradientDICE: Rethinking Generalized Offline Estimation of Stationary Values