Open AccessSoftware crowdsourcing task pricing based on topic model analysis
Author(s) -
Shen YuSong,
Yang Ye,
Wang Yong,
Chang DeLin
Publication year - 2020
Publication title -
iet software
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.305
H-Index - 43
eISSN - 1751-8814
pISSN - 1751-8806
DOI - 10.1049/iet-sen.2019.0168
Subject(s) - crowdsourcing , computer science , task (project management) , crowdsourcing software development , software , machine learning , process (computing) , artificial intelligence , incentive , data mining , data science , software engineering , software development , world wide web , software development process , engineering , systems engineering , programming language , microeconomics , economics , operating system
In software crowdsourcing, task prize is a primary incentive for engaging crowd developers. One of the main challenges in crowdsourcing task pricing is to determine appropriate prizes in order to attract qualified workers. Few studies proposed methods to address this challenge. However, they are either too theoretical or too restricted to be applied for early crowdsourcing planning. In this study, we propose a novel approach, i.e., PTMA, to support early task pricing in software crowdsourcing from textual task requirements. PTMA consists of three phases, namely data pre‐processing, topic extraction, and topic‐based task pricing analysis, integrating 6 machine learning algorithms and 3 analogy‐based models for topic‐based pricing analysis. PTMA is evaluated using data from 2016 software crowdsourcing tasks extracted from TopCoder, the largest software crowdsourcing platform. The results show that: 1) textual requirement information can aid early task pricing in software crowdsourcing; 2) the best predictor in PTMA, based on logistic regression, achieves an accuracy of 88.3% in Pred (30); and 3) PTMA outperforms the existing baseline models by 9% in Pred (30). PTMA greatly simplifies the pricing process by only leveraging textual task description as inputs, and can achieve better prediction accuracy in making task pricing decisions.