English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Policy optimization methods are popular reinforcement learning algorithms,because their incremental and on-policy nature makes them more stable than thevalue-based counterparts. However, the same properties also make them slow toconverge and sample inefficient, as the on-policy requirement precludes datareuse and the incremental updates couple large iteration complexity into thesample complexity. These characteristics have been observed in experiments aswell as in theory in the recent work of~\citet{agarwal2020pc}, which provides apolicy optimization method PCPG that can robustly find near optimal polices forapproximately linear Markov decision processes but suffers from an extremelypoor sample complexity compared with value-based techniques.  In this paper, we propose a new algorithm, COPOE, that overcomes the samplecomplexity issue of PCPG while retaining its robustness to modelmisspecification. Compared with PCPG, COPOE makes several important algorithmicenhancements, such as enabling data reuse, and uses more refined analysistechniques, which we expect to be more broadly applicable to designing newreinforcement learning algorithms. The result is an improvement in samplecomplexity from $\widetilde{O}(1/\epsilon^{11})$ for PCPG to$\widetilde{O}(1/\epsilon^3)$ for PCPG, nearly bridging the gap withvalue-based techniques.

Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximation