Open AccessEmbedding and classifying test execution traces using neural networks
Author(s) -
Tsimpourlas Foivos,
Rooijackers Gwenyth,
Rajan Ajitha,
Allamanis Miltiadis
Publication year - 2022
Publication title -
iet software
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.305
H-Index - 43
eISSN - 1751-8814
pISSN - 1751-8806
DOI - 10.1049/sfw2.12038
Subject(s) - computer science , oracle , artificial neural network , embedding , java , artificial intelligence , machine learning , filter (signal processing) , operating system , programming language , computer vision
Classifying test executions automatically as pass or fail remains a key challenge in software testing and is referred to as the test oracle problem . It is being attempted to solve this problem with supervised learning over test execution traces. A programme is instrumented to gather execution traces as sequences of method invocations. A small fraction of the programme's execution traces is labelled with pass or fail verdicts. Execution traces are then embedded as fixed length vectors and a neural network (NN) component that uses the line‐by‐line information to classify traces as pass or fail is designed. The classification accuracy of this approach is evaluated using subject programs from different application domains—1. Module from Ethereum Blockchain, 2. Module from PyTorch deep learning framework, 3. Microsoft SEAL encryption library components, 4. Sed stream editor, 5. Nine network protocols from Linux packet identifier, L7‐Filter and 6. Utilities library, commons‐lang for Java. For all subject programs, it was found that test execution classification had high precision, recall and specificity, averaging to 93%, 94% and 96%, respectively, while only training with an average 14% of the total traces. Experiments show that the proposed NN‐based approach is promising in classifying test executions from different application domains.