English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent datasets expose the lack of the systematic generalization ability instandard sequence-to-sequence models. In this work, we analyze this behavior ofseq2seq models and identify two contributing factors: a lack of mutualexclusivity bias (i.e., a source sequence already mapped to a target sequenceis less likely to be mapped to other target sequences), and the tendency tomemorize whole examples rather than separating structures from contents. Wepropose two techniques to address these two issues respectively: MutualExclusivity Training that prevents the model from producing seen generationswhen facing novel, unseen examples via an unlikelihood-based loss; andprim2primX data augmentation that automatically diversifies the arguments ofevery syntactic function to prevent memorizing and provide a compositionalinductive bias without exposing test-set data. Combining these two techniques,we show substantial empirical improvements using standard sequence-to-sequencemodels (LSTMs and Transformers) on two widely-used compositionality datasets:SCAN and COGS. Finally, we provide analysis characterizing the improvements aswell as the remaining challenges, and provide detailed ablations of our method.Our code is available at https://github.com/owenzx/met-primaug

Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality