English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite the remarkable evolution of deep neural networks in natural languageprocessing (NLP), their interpretability remains a challenge. Previous worklargely focused on what these models learn at the representation level. Webreak this analysis down further and study individual dimensions (neurons) inthe vector representation learned by end-to-end neural models in NLP tasks. Wepropose two methods: Linguistic Correlation Analysis, based on a supervisedmethod to extract the most relevant neurons with respect to an extrinsic task,and Cross-model Correlation Analysis, an unsupervised method to extract salientneurons w.r.t. the model itself. We evaluate the effectiveness of ourtechniques by ablating the identified neurons and reevaluating the network'sperformance for two tasks: neural machine translation (NMT) and neural languagemodeling (NLM). We further present a comprehensive analysis of neurons with theaim to address the following questions: i) how localized or distributed aredifferent linguistic properties in the models? ii) are certain neuronsexclusive to some properties and not others? iii) is the information more orless distributed in NMT vs. NLM? and iv) how important are the neuronsidentified through the linguistic correlation method to the overall task? Ourcode is publicly available as part of the NeuroX toolkit (Dalvi et al. 2019).

What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models