English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multiple studies have probed representations emerging in neural networkstrained for end-to-end NLP tasks and examined what word-level linguisticinformation may be encoded in the representations. In classical probing, aclassifier is trained on the representations to extract the target linguisticinformation. However, there is a threat of the classifier simply memorizing thelinguistic labels for individual words, instead of extracting the linguisticabstractions from the representations, thus reporting false positive results.While considerable efforts have been made to minimize the memorization problem,the task of actually measuring the amount of memorization happening in theclassifier has been understudied so far. In our work, we propose a simplegeneral method for measuring the memorization effect, based on a symmetricselection of comparable sets of test words seen versus unseen in training. Ourmethod can be used to explicitly quantify the amount of memorization happeningin a probing setup, so that an adequate setup can be chosen and the results ofthe probing can be interpreted with a reliability estimate. We exemplify thisby showcasing our method on a case study of probing for part of speech in atrained neural machine translation encoder.

Measuring Memorization Effect in Word-Level Neural Networks Probing