English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Natural Language Processing (NLP) systems learn harmful societal biases thatcause them to amplify inequality as they are deployed in more and moresituations. To guide efforts at debiasing these systems, the NLP communityrelies on a variety of metrics that quantify bias in models. Some of thesemetrics are intrinsic, measuring bias in word embedding spaces, and some areextrinsic, measuring bias in downstream tasks that the word embeddings enable.Do these intrinsic and extrinsic metrics correlate with each other? We compareintrinsic and extrinsic metrics across hundreds of trained models coveringdifferent tasks and experimental conditions. Our results show no reliablecorrelation between these metrics that holds in all scenarios across tasks andlanguages. We urge researchers working on debiasing to focus on extrinsicmeasures of bias, and to make using these measures more feasible via creationof new challenge sets and annotated test data. To aid this effort, we releasecode, a new intrinsic metric, and an annotated test set focused on gender biasin hate speech.

Intrinsic Bias Metrics Do Not Correlate with Application Bias