English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Toxicity is an increasingly common and severe issue in online spaces.Consequently, a rich line of machine learning research over the past decade hasfocused on computationally detecting and mitigating online toxicity. Theseefforts crucially rely on human-annotated datasets that identify toxic contentof various kinds in social media texts. However, such annotations historicallyyield low inter-rater agreement, which was often dealt with by taking themajority vote or other such approaches to arrive at a single ground truthlabel. Recent research has pointed out the importance of accounting for thesubjective nature of this task when building and utilizing these datasets, andthis has triggered work on analyzing and better understanding raterdisagreements, and how they could be effectively incorporated into the machinelearning developmental pipeline. While these efforts are filling an importantgap, there is a lack of a broader framework about the root causes of raterdisagreement, and therefore, we situate this work within that broaderlandscape. In this survey paper, we analyze a broad set of literature on thereasons behind rater disagreements focusing on online toxicity, and propose adetailed taxonomy for the same. Further, we summarize and discuss the potentialsolutions targeting each reason for disagreement. We also discuss several openissues, which could promote the future development of online toxicity research.

A Taxonomy of Rater Disagreements: Surveying Challenges & Opportunities from the Perspective of Annotating Online Toxicity