Open AccessEvaluation Grade Prediction Method with Limited Information from Experts
Author(s) -
Wei Xiao,
Haotian You,
Jiajun Cheng,
Qiang Gao,
Shanhong Tang
Publication year - 2021
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1757/1/012010
Subject(s) - computer science , set (abstract data type) , artificial intelligence , training set , machine learning , sample (material) , data set , quality (philosophy) , data mining , philosophy , chemistry , epistemology , chromatography , programming language
With the development of artificial intelligence, using machine learning methods to build evaluation models has attracted more and more attention. However, training anevaluation model often needs a lot of labeling samples annotated by experts. It is very difficult to get enough labeling data with a limited group of experts. This paper proposes a method to learn the evaluation model with limited information from experts. This method has two stages. In the first stage, we build a large training set with ordinary people by comparing every two samples. After that, we train a Siamese Network with the paired comparison data set to get a score for each sample. In the second stage, we map the scores to evaluation grades with the help of experts. In the experiments, we use the UCI wine quality data set to evaluate our method. Experimental results demonstrate that we get a basically equivalent accuracy (0.5% decrease) with only1.45% samples labeled byexperts.