English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Instruction tuned large language models (LLMs), such as ChatGPT, demonstrateremarkable performance in a wide range of tasks. Despite numerous recentstudies that examine the performance of instruction-tuned LLMs on various NLPbenchmarks, there remains a lack of comprehensive investigation into theirability to understand cross-lingual sociopragmatic meaning (SM), i.e., meaningembedded within social and interactive contexts. This deficiency arises partlyfrom SM not being adequately represented in any of the existing benchmarks. Toaddress this gap, we present SPARROW, an extensive multilingual benchmarkspecifically designed for SM understanding. SPARROW comprises 169 datasetscovering 13 task types across six primary categories (e.g., anti-sociallanguage detection, emotion recognition). SPARROW datasets encompass 64different languages originating from 12 language families representing 16writing scripts. We evaluate the performance of various multilingual pretrainedlanguage models (e.g., mT5) and instruction-tuned LLMs (e.g., BLOOMZ, ChatGPT)on SPARROW through fine-tuning, zero-shot, and/or few-shot learning. Ourcomprehensive analysis reveals that existing open-source instruction tuned LLMsstill struggle to understand SM across various languages, performing close to arandom baseline in some cases. We also find that although ChatGPT outperformsmany LLMs, it still falls behind task-specific finetuned models with a gap of12.19 SPARROW score. Our benchmark is available at:https://github.com/UBC-NLP/SPARROW

The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages