English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

There is considerable interest in designing meta-reinforcement learning(meta-RL) algorithms, which enable autonomous agents to adapt new tasks fromsmall amount of experience. In meta-RL, the specification (such as rewardfunction) of current task is hidden from the agent. In addition, states arehidden within each task owing to sensor noise or limitations in realisticenvironments. Therefore, the meta-RL agent faces the challenge of specifyingboth the hidden task and states based on small amount of experience. To addressthis, we propose estimating disentangled belief about task and states,leveraging an inductive bias that the task and states can be regarded as globaland local features of each task. Specifically, we train a hierarchicalstate-space model (HSSM) parameterized by deep neural networks as anenvironment model, whose global and local latent variables correspond to taskand states, respectively. Because the HSSM does not allow analyticalcomputation of posterior distribution, i.e., belief, we employ amortizedinference to approximate it. After the belief is obtained, we can augmentobservations of a model-free policy with the belief to efficiently train thepolicy. Moreover, because task and state information are factorized andinterpretable, the downstream policy training is facilitated compared with theprior methods that did not consider the hierarchical nature. Empiricalvalidations on a GridWorld environment confirm that the HSSM can separate thehidden task and states information. Then, we compare the meta-RL agent with theHSSM to prior meta-RL methods in MuJoCo environments, and confirm that ouragent requires less training data and reaches higher final performance.

Estimating Disentangled Belief about Hidden State and Hidden Task for Meta-RL