English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Since the popularization of the Transformer as a general-purpose featureencoder for NLP, many studies have attempted to decode linguistic structurefrom its novel multi-head attention mechanism. However, much of such workfocused almost exclusively on English -- a language with rigid word order and alack of inflectional morphology. In this study, we present decoding experimentsfor multilingual BERT across 18 languages in order to test the generalizabilityof the claim that dependency syntax is reflected in attention patterns. We showthat full trees can be decoded above baseline accuracy from single attentionheads, and that individual relations are often tracked by the same heads acrosslanguages. Furthermore, in an attempt to address recent debates about thestatus of attention as an explanatory mechanism, we experiment with fine-tuningmBERT on a supervised parsing objective while freezing different series ofparameters. Interestingly, in steering the objective to learn explicitlinguistic structure, we find much of the same structure represented in theresulting attention patterns, with interesting differences with respect towhich parameters are frozen.

Attention Can Reflect Syntactic Structure (If You Let It)