Open AccessTest Input Selection for Deep Neural Networks
Author(s) -
Zhiyu Wang,
Sihan Xu,
Xiangrui Cai,
Hua Ji
Publication year - 2020
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1693/1/012017
Subject(s) - oracle , computer science , test suite , artificial intelligence , machine learning , deep learning , selection (genetic algorithm) , test (biology) , software , quality (philosophy) , artificial neural network , suite , regression testing , test case , data mining , software system , software engineering , software construction , operating system , history , paleontology , philosophy , regression analysis , archaeology , epistemology , biology
With the rapid development of deep learning technologies, the quality and security of deep learning systems have aroused great concern recently. Much research has been done in testing deep learning systems. Nevertheless, the oracle problem still remains since the input space of DNN-based software is usually very large and manually labeling is boring and cost-effective. Inspired by structural testing for traditional software, in this paper, we propose an algorithm to mitigate the oracle problem by selecting test inputs worth labeling. Specifically, we propose a test subset selection algorithm that can automatically select a test suite with high coverage but a small size when the labeling budget is limited. Compared with DeepXplore, the proposed algorithm can generate smaller test suites with higher coverage.