English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We study Policy-extended Value Function Approximator (PeVFA) in ReinforcementLearning (RL), which extends conventional value function approximator (VFA) totake as input not only the state (and action) but also an explicit policyrepresentation. Such an extension enables PeVFA to preserve values of multiplepolicies at the same time and brings an appealing characteristic, i.e.,\emph{value generalization among policies}. We formally analyze the valuegeneralization under Generalized Policy Iteration (GPI). From theoretical andempirical lens, we show that generalized value estimates offered by PeVFA mayhave lower initial approximation error to true values of successive policies,which is expected to improve consecutive value approximation during GPI. Basedon above clues, we introduce a new form of GPI with PeVFA which leverages thevalue generalization along policy improvement path. Moreover, we propose arepresentation learning framework for RL policy, providing several approachesto learn effective policy embeddings from policy network parameters orstate-action pairs. In our experiments, we evaluate the efficacy of valuegeneralization offered by PeVFA and policy representation learning in severalOpenAI Gym continuous control tasks. For a representative instance of algorithmimplementation, Proximal Policy Optimization (PPO) re-implemented under theparadigm of GPI with PeVFA achieves about 40\% performance improvement on itsvanilla counterpart in most environments.

What About Inputing Policy in Value Function: Policy Representation and Policy-extended Value Function Approximator