English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We investigate the extent to which modern, neural language models aresusceptible to structural priming, the phenomenon whereby the structure of asentence makes the same structure more probable in a follow-up sentence. Weexplore how priming can be used to study the potential of these models to learnabstract structural information, which is a prerequisite for good performanceon tasks that require natural language understanding skills. We introduce anovel metric and release Prime-LM, a large corpus where we control for variouslinguistic factors which interact with priming strength. We find thatTransformer models indeed show evidence of structural priming, but also thatthe generalisations they learned are to some extent modulated by semanticinformation. Our experiments also show that the representations acquired by themodels may not only encode abstract sequential structure but involve certainlevel of hierarchical syntactic information. More generally, our study showsthat the priming paradigm is a useful, additional tool for gaining insightsinto the capacities of language models and opens the door to futurepriming-based investigations that probe the model's internal states.

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations