English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained multilingual language models such as mBERT and XLM-R havedemonstrated great potential for zero-shot cross-lingual transfer to lowweb-resource languages (LRL). However, due to limited model capacity, the largedifference in the sizes of available monolingual corpora between highweb-resource languages (HRL) and LRLs does not provide enough scope ofco-embedding the LRL with the HRL, thereby affecting downstream taskperformance of LRLs. In this paper, we argue that relatedness among languagesin a language family along the dimension of lexical overlap may be leveraged toovercome some of the corpora limitations of LRLs. We propose Overlap BPE(OBPE), a simple yet effective modification to the BPE vocabulary generationalgorithm which enhances overlap across related languages. Through extensiveexperiments on multiple NLP tasks and datasets, we observe that OBPE generatesa vocabulary that increases the representation of LRLs via tokens shared withHRLs. This results in improved zero-shot transfer from related HRLs to LRLswithout reducing HRL representation and accuracy. Unlike previous studies thatdismissed the importance of token-overlap, we show that in the low-resourcerelated language setting, token overlap matters. Synthetically reducing theoverlap to zero can cause as much as a four-fold drop in zero-shot transferaccuracy.

Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages