English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Discriminatory social biases, including gender biases, have been found inPre-trained Language Models (PLMs). In Natural Language Inference (NLI), recentbias evaluation methods have observed biased inferences from the outputs of aparticular label such as neutral or entailment. However, since different biasedinferences can be associated with different output labels, it is inaccurate fora method to rely on one label. In this work, we propose an evaluation methodthat considers all labels in the NLI task. We create evaluation data and assignthem into groups based on their expected biased output labels. Then, we definea bias measure based on the corresponding label output of each data group. Inthe experiment, we propose a meta-evaluation method for NLI bias measures, andthen use it to confirm that our measure can evaluate bias more accurately thanthe baseline. Moreover, we show that our evaluation method is applicable tomultiple languages by conducting the meta-evaluation on PLMs in three differentlanguages: English, Japanese, and Chinese. Finally, we evaluate PLMs of eachlanguage to confirm their bias tendency. To our knowledge, we are the first tobuild evaluation datasets and measure the bias of PLMs from the NLI task inJapanese and Chinese.

Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels