Teaching Black Box Testing Techniques Through Specification Patterns

Software verification is one of the most important activities in the software development cycle, and testing remains the most common approach to verification used in industry. The goal of blackbox testing (functional testing) is to verify the system’s adherence to specifications. The notion of patterns and scopes developed by Dwyer et al. provides a cohesive and rich set of examples to teach black-box testing strategies. A pattern describes a recurring software property, and a scope specifies the interval of program execution where a pattern must hold. A property specified using a pattern and scope combination has characteristics that must be satisfied if it is to hold. Based on these characteristics, there is a large set of behaviors that can be examined using black-box testing techniques. In a complementary fashion, the behaviors specified by patterns and scopes provide clear and simple examples that can enhance the understanding of these testing techniques. In this paper, we describe an approach and present general lessons and exercises that demonstrate how patterns and scopes can be used to teach boundary value analysis and equivalence class testing, which are two of the most commonly used black-box testing techniques. As a side effect of this approach, students are exposed to, and become familiar with, formally specifying system behavior.