English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recurrent Neural Networks (RNNs) trained on a language modeling task havebeen shown to acquire a number of non-local grammatical dependencies with somesuccess. Here, we provide new evidence that RNN language models are sensitiveto hierarchical syntactic structure by investigating the filler--gap dependencyand constraints on it, known as syntactic islands. Previous work isinconclusive about whether RNNs learn to attenuate their expectations for gapsin island constructions in particular or in any sufficiently complex syntacticenvironment. This paper gives new evidence for the former by providing controlstudies that have been lacking so far. We demonstrate that two state-of-the-artRNN models are are able to maintain the filler--gap dependency throughunbounded sentential embeddings and are also sensitive to the hierarchicalrelationship between the filler and the gap. Next, we demonstrate that themodels are able to maintain possessive pronoun gender expectations throughisland constructions---this control case rules out the possibility that islandconstructions block all information flow in these networks. We also evaluatethree untested islands constraints: coordination islands, left branch islands,and sentential subject islands. Models are able to learn left branch islandsand learn coordination islands gradiently, but fail to learn sentential subjectislands. Through these controls and new tests, we provide evidence that modelbehavior is due to finer-grained expectations than gross syntactic complexity,but also that the models are conspicuously un-humanlike in some of theirperformance characteristics.

What Syntactic Structures block Dependencies in RNN Language Models?