We need more coverage, stat! classroom experience with the software ICU

For empirical software engineering to reach its fullest potential, we must develop effective, experiential approaches to learning about it in a classroom setting. In this paper, we report on a case study involving a new approach to classroom-based empirical software engineering called the “Software ICU”. In this approach, students learn about nine empirical project “vital signs” and use the Hackys-tat Framework to put their projects into a virtual “intensive care unit” where these vital signs can be assessed and monitored. We used both questionnaire and log data to gain insight into the strengths and weaknesses of this approach. Our evaluation provides both quantitative and qualitative evidence concerning the overhead of the system; the relative utility of different vital signs; the frequency of use; and the perceived appropriateness outside of the classroom setting. In addition to benefits, we found evidence of measurement dysfunction induced directly by the presence of the Software ICU. We compare these results to case studies we performed in 2003 and 2006 using the Hackystat Framework but not the Software ICU. We use these findings to orient future research on empirical software engineering both inside and outside of the classroom.