Premium
Machine learning from crowds: A systematic review of its applications
Author(s) -
G. Rodrigo Enrique,
Aledo Juan A.,
Gámez José A.
Publication year - 2018
Publication title -
wiley interdisciplinary reviews: data mining and knowledge discovery
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.506
H-Index - 47
eISSN - 1942-4795
pISSN - 1942-4787
DOI - 10.1002/widm.1288
Subject(s) - crowdsourcing , crowds , computer science , field (mathematics) , data science , machine learning , artificial intelligence , variety (cybernetics) , forcing (mathematics) , key (lock) , quality (philosophy) , world wide web , computer security , philosophy , mathematics , epistemology , climatology , pure mathematics , geology
Crowdsourcing opens the door to solving a wide variety of problems that previously were unfeasible in the field of machine learning, allowing us to obtain relatively low cost labeled data in a small amount of time. However, due to the uncertain quality of labelers, the data to deal with are sometimes unreliable, forcing practitioners to collect information redundantly, which poses new challenges in the field. Despite these difficulties, many applications of machine learning using crowdsourced data have recently been published that achieved state of the art results in relevant problems. We have analyzed these applications following a systematic methodology, classifying them into different fields of study, highlighting several of their characteristics and showing the recent interest in the use of crowdsourcing for machine learning. We also identify several exciting research lines based on the problems that remain unsolved to foster future research in this field. This article is categorized under: Technologies > Machine Learning Application Areas > Science and Technology Fundamental Concepts of Data and Knowledge > Key Design Issues in Data Mining