Product Focused Software Process Improvement

Software process improvement efforts often seek to shorten development lead-time. A potential means is to facilitate architectural changes by providing a design rationale, i.e. a documentation of why the architecture is built as it is. The hypothesis is that changes will be faster and more correct if such information is available during change impact analysis. This paper presents a controlled experiment where the value of having access to a retrospective design rationale is evaluated both quantitatively and qualitatively. Realistic change tasks are applied by 17 subjects from both industry and academia on two complex systems from the domain of embedded real-time systems. The results from the quantitative analysis show that, for one of the systems, there is a significant improvement in correctness and speed when subjects have access to a design rationale document. In the qualitative analysis, design rationale was considered helpful for speeding up changes and improving correctness. For the other system the results were inconclusive, and further studies are recommended in order to increase the understanding of the role of a design rationale in architectural evolution of software systems.