English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As language models (LMs) become increasingly powerful and widely used, it isimportant to quantify them for sociodemographic bias with potential for harm.Prior measures of bias are sensitive to perturbations in the templates designedto compare performance across social groups, due to factors such as lowdiversity or limited number of templates. Also, most previous work considersonly one NLP task. We introduce Comprehensive Assessment of Language Models(CALM) for robust measurement of two types of universally relevantsociodemographic bias, gender and race. CALM integrates sixteen datasets forquestion-answering, sentiment analysis and natural language inference. Examplesfrom each dataset are filtered to produce 224 templates with high diversity(e.g., length, vocabulary). We assemble 50 highly frequent person names foreach of seven distinct demographic groups to generate 78,400 prompts coveringthe three NLP tasks. Our empirical evaluation shows that CALM bias scores aremore robust and far less sensitive than previous bias measurements toperturbations in the templates, such as synonym substitution, or to randomsubset selection of templates. We apply CALM to 20 large language models, andfind that for 2 language model series, larger parameter models tend to be morebiased than smaller ones. The T0 series is the least biased model families, ofthe 20 LLMs investigated here. The code is available athttps://github.com/vipulgupta1011/CALM.

CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Bias