English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

An exciting advancement in the field of multilingual models is the emergenceof autoregressive models with zero- and few-shot capabilities, a phenomenonwidely reported in large-scale language models. To further improve modeladaptation to cross-lingual tasks, another trend is to further fine-tune thelanguage models with either full fine-tuning or parameter-efficient tuning.However, the interaction between parameter-efficient fine-tuning (PEFT) andcross-lingual tasks in multilingual autoregressive models has yet to bestudied. Specifically, we lack an understanding of the role of linguisticdistributions in multilingual models in the effectiveness of token-based prompttuning. To address this question, we conduct experiments comparing prompttuning and fine-tuning on the decoder-based multilingual model, XGLM, with fourcross-lingual tasks (XNLI, PAWS-X, POS, NER). According to our study, prompttuning achieves on par or better performance over fine-tuning across alllanguages while updating at most 0.13\% of the model parameters. Moreover, weempirically show that prompt tuning is more effective in enhancing theperformance of low-resource languages than fine-tuning. Our further analysisshows that the phenomenon is related to the tokenization scheme of themultilingual model.

On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Model