English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Compositionality, the phenomenon where the meaning of a phrase can be derivedfrom its constituent parts, is a hallmark of human language. At the same time,many phrases are non-compositional, carrying a meaning beyond that of each partin isolation. Representing both of these types of phrases is critical forlanguage understanding, but it is an open question whether modern languagemodels (LMs) learn to do so; in this work we examine this question. We firstformulate a problem of predicting the LM-internal representations of longerphrases given those of their constituents. We find that the representation of aparent phrase can be predicted with some accuracy given an affinetransformation of its children. While we would expect the predictive accuracyto correlate with human judgments of semantic compositionality, we find this islargely not the case, indicating that LMs may not accurately distinguishbetween compositional and non-compositional phrases. We perform a variety ofanalyses, shedding light on when different varieties of LMs do and do notgenerate compositional representations, and discuss implications for futuremodeling work.

Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models