English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Natural language instruction following is paramount to enable collaborationbetween artificial agents and human beings. Natural language-conditionedreinforcement learning (RL) agents have shown how natural languages'properties, such as compositionality, can provide a strong inductive bias tolearn complex policies. Previous architectures like HIGhER combine the benefitof language-conditioning with Hindsight Experience Replay (HER) to deal withsparse rewards environments. Yet, like HER, HIGhER relies on an oraclepredicate function to provide a feedback signal highlighting which linguisticdescription is valid for which state. This reliance on an oracle limits itsapplication. Additionally, HIGhER only leverages the linguistic informationcontained in successful RL trajectories, thus hurting its final performance anddata-efficiency. Without early successful trajectories, HIGhER is no betterthan DQN upon which it is built. In this paper, we propose the Emergent TextualHindsight Experience Replay (ETHER) agent, which builds on HIGhER and addressesboth of its limitations by means of (i) a discriminative visual referentialgame, commonly studied in the subfield of Emergent Communication (EC), usedhere as an unsupervised auxiliary task and (ii) a semantic grounding scheme toalign the emergent language with the natural language of theinstruction-following benchmark. We show that the referential game's agentsmake an artificial language emerge that is aligned with the natural-likelanguage used to describe goals in the BabyAI benchmark and that it isexpressive enough so as to also describe unsuccessful RL trajectories and thusprovide feedback to the RL agent to leverage the linguistic, structuredinformation contained in all trajectories. Our work shows that EC is a viableunsupervised auxiliary task for RL and provides missing pieces to make HER morewidely applicable.

ETHER: Aligning Emergent Communication for Hindsight Experience Replay