English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Multilingual models are often particularly dependent on scaling to generalizeto a growing number of languages. Compression techniques are widely relied uponto reconcile the growth in model size with real world resource constraints, butcompression can have a disparate effect on model performance for low-resourcelanguages. It is thus crucial to understand the trade-offs between scale,multilingualism, and compression. In this work, we propose an experimentalframework to characterize the impact of sparsifying multilingual pre-trainedlanguage models during fine-tuning. Applying this framework to mBERT namedentity recognition models across 40 languages, we find that compression confersseveral intriguing and previously unknown generalization properties. Incontrast to prior findings, we find that compression may improve modelrobustness over dense models. We additionally observe that under certainsparsification regimes compression may aid, rather than disproportionatelyimpact the performance of low-resource languages.

Intriguing Properties of Compression on Multilingual Models