English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Learning to evaluate and improve policies is a core problem of ReinforcementLearning (RL). Traditional RL algorithms learn a value function defined for asingle policy. A recently explored competitive alternative is to learn a singlevalue function for many policies. Here we combine the actor-critic architectureof Parameter-Based Value Functions and the policy embedding of PolicyEvaluation Networks to learn a single value function for evaluating (and thushelping to improve) any policy represented by a deep neural network (NN). Themethod yields competitive experimental results. In continuous control problemswith infinitely many states, our value function minimizes its prediction errorby simultaneously learning a small set of `probing states' and a mapping fromactions produced in probing states to the policy's return. The method extractscrucial abstract knowledge about the environment in form of very few statessufficient to fully specify the behavior of many policies. A policy improvessolely by changing actions in probing states, following the gradient of thevalue function's predictions. Surprisingly, it is possible to clone thebehavior of a near-optimal policy in Swimmer-v3 and Hopper-v3 environments onlyby knowing how to act in 3 and 5 such learned states, respectively. Remarkably,our value function trained to evaluate NN policies is also invariant to changesof the policy architecture: we show that it allows for zero-shot learning oflinear policies competitive with the best policy seen during training. Our codeis public.

General Policy Evaluation and Improvement by Learning to Identify Few But Crucial States