English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The global surge in AI applications is transforming industries, leading todisplacement and complementation of existing jobs, while also giving rise tonew employment opportunities. Human oversight of AI is an emerging task inwhich human workers interact with an AI model to improve its performance,safety, and compliance with normative principles. Data annotation, encompassingthe labelling of images or annotating of texts, serves as a critical humanoversight process, as the quality of a dataset directly influences the qualityof AI models trained on it. Therefore, the efficiency of human oversight workstands as an important competitive advantage for AI developers. This paperdelves into the foundational economics of human oversight, with a specificfocus on the impact of norm design and monetary incentives on data quality andcosts. An experimental study involving 307 data annotators examines six groupswith varying task instructions (norms) and monetary incentives. Results revealthat annotators provided with clear rules exhibit higher accuracy rates,outperforming those with vague standards by 14%. Similarly, annotatorsreceiving an additional monetary incentive perform significantly better, withthe highest accuracy rate recorded in the group working with both clear rulesand incentives (87.5% accuracy). However, both groups require more time tocomplete tasks, with a 31% increase in average task completion time compared tothose working with standards and no incentives. These empirical findingsunderscore the trade-off between data quality and efficiency in data curation,shedding light on the nuanced impact of norm design and incentives on theeconomics of AI development. The paper contributes experimental insights todiscussions on the economical, ethical, and legal considerations of AItechnologies.

The Economics of Human Oversight: How Norms and Incentives Affect Costs and Performance of AI Workers