English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Previous work suggests that RNNs trained on natural language corpora cancapture number agreement well for simple sentences but perform less well whensentences contain agreement attractors: intervening nouns between the verb andthe main subject with grammatical number opposite to the latter. This suggeststhese models may not learn the actual syntax of agreement, but rather infershallower heuristics such as `agree with the recent noun'. In this work, weinvestigate RNN models with varying inductive biases trained on selectivelychosen `hard' agreement instances, i.e., sentences with at least one agreementattractor. For these the verb number cannot be predicted using a simple linearheuristic, and hence they might help provide the model additional cues forhierarchical syntax. If RNNs can learn the underlying agreement rules whentrained on such hard instances, then they should generalize well to othersentences, including simpler ones. However, we observe that several RNN types,including the ONLSTM which has a soft structural inductive bias, surprisinglyfail to perform well on sentences without attractors when trained solely onsentences with attractors. We analyze how these selectively trained RNNscompare to the baseline (training on a natural distribution of agreementattractors) along the dimensions of number agreement accuracy, representationalsimilarity, and performance across different syntactic constructions. Ourfindings suggest that RNNs trained on our hard agreement instances still do notcapture the underlying syntax of agreement, but rather tend to overfit thetraining distribution in a way which leads them to perform poorly on `easy'out-of-distribution instances. Thus, while RNNs are powerful models which canpick up non-trivial dependency patterns, inducing them to do so at the level ofsyntax rather than surface remains a challenge.

Can RNNs trained on harder subject-verb agreement instances still perform well on easier ones?