English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent studies have shown that sequence-to-sequence (seq2seq) models strugglewith compositional generalization (CG), i.e., the ability to systematicallygeneralize to unseen compositions of seen components. There is mountingevidence that one of the reasons hindering CG is the representation of theencoder uppermost layer is entangled, i.e., the syntactic and semanticrepresentations of sequences are entangled. However, we consider that thepreviously identified representation entanglement problem is not comprehensiveenough. Additionally, we hypothesize that the source keys and valuesrepresentations passing into different decoder layers are also entangled.Starting from this intuition, we propose \textsc{CompoSition} (\textbf{Compo}se\textbf{S}yntactic and Semant\textbf{i}c Representa\textbf{tion}s), anextension to seq2seq models which learns to compose representations ofdifferent encoder layers dynamically for different tasks, since recent studiesreveal that the bottom layers of the Transformer encoder contain more syntacticinformation and the top ones contain more semantic information. Specifically,we introduce a \textit{composed layer} between the encoder and decoder tocompose different encoder layers' representations to generate specific keys andvalues passing into different decoder layers. \textsc{CompoSition} achievescompetitive results on two comprehensive and realistic benchmarks, whichempirically demonstrates the effectiveness of our proposal. Codes are availableat~\url{https://github.com/thinkaboutzero/COMPOSITION}.

Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalization