English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recent years, machine learning (ML) has relied heavily on crowdworkersboth for building datasets and for addressing research questions requiringhuman interaction or judgment. The diverse tasks performed and uses of the dataproduced render it difficult to determine when crowdworkers are best thought ofas workers (versus human subjects). These difficulties are compounded byconflicting policies, with some institutions and researchers regarding all MLcrowdworkers as human subjects and others holding that they rarely constitutehuman subjects. Notably few ML papers involving crowdwork mention IRBoversight, raising the prospect of non-compliance with ethical and regulatoryrequirements. We investigate the appropriate designation of ML crowdsourcingstudies, focusing our inquiry on natural language processing to expose uniquechallenges for research oversight. Crucially, under the U.S. Common Rule, thesejudgments hinge on determinations of aboutness, concerning both whom (or what)the collected data is about and whom (or what) the analysis is about. Wehighlight two challenges posed by ML: the same set of workers can servemultiple roles and provide many sorts of information; and ML research tends toembrace a dynamic workflow, where research questions are seldom stated ex anteand data sharing opens the door for future studies to aim questions atdifferent targets. Our analysis exposes a potential loophole in the CommonRule, where researchers can elude research ethics oversight by splitting datacollection and analysis into distinct studies. Finally, we offer several policyrecommendations to address these concerns.

Resolving the Human Subjects Status of Machine Learning's Crowdworkers