English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Natural Language Inference (NLI) has been a cornerstone task in evaluatinglanguage models' inferential reasoning capabilities. However, the standardthree-way classification scheme used in NLI has well-known shortcomings inevaluating models' ability to capture the nuances of natural human reasoning.In this paper, we argue that the operationalization of the neutral label incurrent NLI datasets has low validity, is interpreted inconsistently, and thatat least one important sense of neutrality is often ignored. We uncover thedetrimental impact of these shortcomings, which in some cases leads toannotation datasets that actually decrease performance on downstream tasks. Wecompare approaches of handling annotator disagreement and identify flaws in arecent NLI dataset that designs an annotator study based on a problematicoperationalization. Our findings highlight the need for a more refinedevaluation framework for NLI, and we hope to spark further discussion andaction in the NLP community.

No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inference