English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Model-free deep reinforcement learning (RL) methods have been successful in awide variety of simulated domains. However, a major obstacle facing deep RL inthe real world is their high sample complexity. Batch policy gradient methodsoffer stable learning, but at the cost of high variance, which often requireslarge batches. TD-style methods, such as off-policy actor-critic andQ-learning, are more sample-efficient but biased, and often require costlyhyperparameter sweeps to stabilize. In this work, we aim to develop methodsthat combine the stability of policy gradients with the efficiency ofoff-policy RL. We present Q-Prop, a policy gradient method that uses a Taylorexpansion of the off-policy critic as a control variate. Q-Prop is both sampleefficient and stable, and effectively combines the benefits of on-policy andoff-policy methods. We analyze the connection between Q-Prop and existingmodel-free algorithms, and use control variate theory to derive two variants ofQ-Prop with conservative and aggressive adaptation. We show that conservativeQ-Prop provides substantial gains in sample efficiency over trust region policyoptimization (TRPO) with generalized advantage estimation (GAE), and improvesstability over deep deterministic policy gradient (DDPG), the state-of-the-arton-policy and off-policy methods, on OpenAI Gym's MuJoCo continuous controlenvironments.

Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic