Case-based reasoning vs parametric models for software quality optimization

Background: There are many data mining methods but few comparisons between them. For example, there are at least two ways to build quality optimizers, programs that find project options that change quality measures like defects, development effort (total staff hours), and time (elapsed calendar months). In the first way, we construct a parametric model to represent prior software projects. In the second way, we just apply case-based reasoning to reason directly from historical cases. Aim: To assess case-based reasoning vs parametric modeling for quality optimization. Method: We compared the W case-based reasoner against the SEEWAW parametric modeling tool. Results: W is easy to explain and fast to build. It makes no parametric assumptions and hence can be rapidly applied to project data in many formats. SEESAW is an elaborate tool that can only process project data expressed in a particular ontology (i.e. just the COCOMO attributes). It is also slower to execute than W. In 24 different tests comparing W and SEESAW, W always performs at least as well as SEESAW. In 6 of those tests W performed statistically better (all tests used Mann-Whitney, 95% confidence). Lastly, like any CBR method, it comes with a built-in maintenance strategy (just add more cases). Conclusion: The W case-based reasoning tool is recommended over the SEESAW parametric modeling tool for purposes of quality optimization (except in the case where there is no local data).