English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural networks are very powerful learning systems, but they do not readilygeneralize from one task to the other. This is partly due to the fact that theydo not learn in a compositional way, that is, by discovering skills that areshared by different tasks, and recombining them to solve new problems. In thispaper, we explore the compositional generalization capabilities of recurrentneural networks (RNNs). We first propose the lookup table composition domain asa simple setup to test compositional behaviour and show that it istheoretically possible for a standard RNN to learn to behave compositionally inthis domain when trained with standard gradient descent and provided withadditional supervision. We then remove this additional supervision and performa search over a large number of model initializations to investigate theproportion of RNNs that can still converge to a compositional solution. Wediscover that a small but non-negligible proportion of RNNs do reach partialcompositional solutions even without special architectural constraints. Thissuggests that a combination of gradient descent and evolutionary strategiesdirectly favouring the minority models that developed more compositionalapproaches might suffice to lead standard RNNs towards compositional solutions.

Memorize or generalize? Searching for a compositional RNN in a haystack