English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Crowdsourced annotation is vital to both collecting labelled data to trainand test automated content moderation systems and to support human-in-the-loopreview of system decisions. However, annotation tasks such as judging hatespeech are subjective and thus highly sensitive to biases stemming fromannotator beliefs, characteristics and demographics. We conduct twocrowdsourcing studies on Mechanical Turk to examine annotator bias in labellingsexist and misogynistic hate speech. Results from 109 annotators show thatannotator political inclination, moral integrity, personality traits, andsexist attitudes significantly impact annotation accuracy and the tendency totag content as hate speech. In addition, semi-structured interviews with ninecrowd workers provide further insights regarding the influence of subjectivityon annotations. In exploring how workers interpret a task - shaped by complexnegotiations between platform structures, task instructions, subjectivemotivations, and external contextual factors - we see annotations not onlyimpacted by worker factors but also simultaneously shaped by the structuresunder which they labour.

How Crowd Worker Factors Influence Subjective Annotations: A Study of Tagging Misogynistic Hate Speech in Tweets