English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Crowdsourcing provides an exceptional opportunity for the rapid collection of human input for data acquisition and labelling. This approach have been adopted in multiple domains and researchers are now able to reach a demographically diverse audience at low cost. However, it remains the question of whether the results are still valid and reliable. Previous work have introduced different mechanisms to ensure data reliability in crowdsourcing. This work examines to which extend, "trapping question" or "outliers detection" assure reliable results to the detriment of, overloading task content with stimuli that are not of interest for the researcher, or by discarding data points that might be the true opinion of a worker. To this end, a speech quality assessment study have been conducted in a web crowdsourcing platform, following the ITU-T Rec. P.800. Workers assessed the speech stimuli of the database 501 from the ITU-T Rec. P.863. We examine results' validity in terms of correlations to previous ratings collected in laboratory. Our outcomes shows that neither of the techniques under investigation improve results accuracy by itself, but a combination of both. Our goal is to provide empirical guidance for designing experiments in crowdsourcing while ensuring data reliability.

Outliers Detection vs. Control Questions to Ensure Reliable Results in Crowdsourcing.