English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep reinforcement learning (RL) algorithms have achieved great success on awide variety of sequential decision-making tasks. However, many of thesealgorithms suffer from high sample complexity when learning from scratch usingenvironmental rewards, due to issues such as credit-assignment andhigh-variance gradients, among others. Transfer learning, in which knowledgegained on a source task is applied to more efficiently learn a different butrelated target task, is a promising approach to improve the sample complexityin RL. Prior work has considered using pre-trained teacher policies to enhancethe learning of the student policy, albeit with the constraint that the teacherand the student MDPs share the state-space or the action-space. In this paper,we propose a new framework for transfer learning where the teacher and thestudent can have arbitrarily different state- and action-spaces. To handle thismismatch, we produce embeddings which can systematically extract knowledge fromthe teacher policy and value networks, and blend it into the student networks.To train the embeddings, we use a task-aligned loss and show that therepresentations could be enriched further by adding a mutual information loss.Using a set of challenging simulated robotic locomotion tasks involvingmany-legged centipedes, we demonstrate successful transfer learning insituations when the teacher and student have different state- andaction-spaces.

Mutual Information Based Knowledge Transfer Under State-Action Dimension Mismatch