English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained multilingual language models underpin a large portion of modernNLP tools outside of English. A strong baseline for specializing these modelsfor specific languages is Language-Adaptive Pre-Training (LAPT). However,retaining a large cross-lingual vocabulary and embedding matrix comes atconsiderable excess computational cost during adaptation. In this study, wepropose several simple techniques to replace a cross-lingual vocabulary with acompact, language-specific one. Namely, we address strategies forre-initializing the token embedding matrix after vocabulary specialization. Wethen provide a systematic experimental comparison of our techniques, inaddition to the recently-proposed Focus method. We demonstrate that: 1)Embedding-replacement techniques in the monolingual transfer literature areinadequate for adapting multilingual models. 2) Replacing cross-lingualvocabularies with smaller specialized ones provides an efficient method toimprove performance in low-resource languages. 3) Simple embeddingre-initialization techniques based on script-wise sub-distributions rivaltechniques such as Focus, which rely on similarity scores obtained from anauxiliary model.

Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages