English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

The global surge in AI applications is transforming industries, leading todisplacement and complementation of existing jobs, while also giving rise tonew employment opportunities. Data annotation, encompassing the labelling ofimages or annotating of texts by human workers, crucially influences thequality of a dataset directly influences the quality of AI models trained onit. This paper delves into the economics of data annotation, with a specificfocus on the impact of task instruction design (that is, the choice betweenrules and standards as theorised in law and economics) and monetary incentiveson data quality and costs. An experimental study involving 307 data annotatorsexamines six groups with varying task instructions (norms) and monetaryincentives. Results reveal that annotators provided with clear rules exhibithigher accuracy rates, outperforming those with vague standards by 14%.Similarly, annotators receiving an additional monetary incentive performsignificantly better, with the highest accuracy rate recorded in the groupworking with both clear rules and incentives (87.5% accuracy). In addition, ourresults show that rules are perceived as being more helpful by annotators thanstandards and reduce annotators' difficulty in annotating images. Theseempirical findings underscore the double benefit of rule-based instructions onboth data quality and worker wellbeing. Our research design allows us to revealthat, in our study, rules are more cost-efficient in increasing accuracy thanmonetary incentives. The paper contributes experimental insights to discussionson the economical, ethical, and legal considerations of AI technologies.Addressing policymakers and practitioners, we emphasise the need for a balancedapproach in optimising data annotation processes for efficient and ethical AIdevelopment and usage.

Improving Task Instructions for Data Annotators: How Clear Rules and Higher Pay Increase Performance in Data Annotation in the AI Economy