English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

How does the input segmentation of pretrained language models (PLMs) affecttheir interpretations of complex words? We present the first studyinvestigating this question, taking BERT as the example PLM and focusing on itssemantic representations of English derivatives. We show that PLMs can beinterpreted as serial dual-route models, i.e., the meanings of complex wordsare either stored or else need to be computed from the subwords, which impliesthat maximally meaningful input tokens should allow for the best generalizationon new words. This hypothesis is confirmed by a series of semantic probingtasks on which DelBERT (Derivation leveraging BERT), a model with derivationalinput segmentation, substantially outperforms BERT with WordPiece segmentation.Our results suggest that the generalization capabilities of PLMs could befurther improved if a morphologically-informed vocabulary of input tokens wereused.

Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words