English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Lying on the heart of intelligent decision-making systems, how policy isrepresented and optimized is a fundamental problem. The root challenge in thisproblem is the large scale and the high complexity of policy space, whichexacerbates the difficulty of policy learning especially in real-worldscenarios. Towards a desirable surrogate policy space, recently policyrepresentation in a low-dimensional latent space has shown its potential inimproving both the evaluation and optimization of policy. The key questioninvolved in these studies is by what criterion we should abstract the policyspace for desired compression and generalization. However, both the theory onpolicy abstraction and the methodology on policy representation learning areless studied in the literature. In this work, we make very first efforts tofill up the vacancy. First, we propose a unified policy abstraction theory,containing three types of policy abstraction associated to policy features atdifferent levels. Then, we generalize them to three policy metrics thatquantify the distance (i.e., similarity) of policies, for more convenient usein learning policy representation. Further, we propose a policy representationlearning approach based on deep metric learning. For the empirical study, weinvestigate the efficacy of the proposed policy metrics and representations, incharacterizing policy difference and conveying policy generalizationrespectively. Our experiments are conducted in both policy optimization andevaluation problems, containing trust-region policy optimization (TRPO),diversity-guided evolution strategy (DGES) and off-policy evaluation (OPE).Somewhat naturally, the experimental results indicate that there is no auniversally optimal abstraction for all downstream learning problems; while theinfluence-irrelevance policy abstraction can be a generally preferred choice.

Towards A Unified Policy Abstraction Theory and Representation Learning Approach in Markov Decision Processes