Enhancing Verification and Validation Education Using Active Learning Tools Developed through an Academia-Industry Partnership

Imparting real world experiences in a software verification and validation (SV&V) course is often a challenge due to the lack of effective active learning tools. This pedagogical requirement is important because graduates are expected to develop software that meets rigorous quality standards in functional and application domains. Realizing the necessity of such teaching tools, the authors designed and developed eighteen (18) delivery hours of Case Studies, sixteen (16) delivery hours of Class Exercises, and six (6) delivery hours of Video Case Studies for use in courses that impart knowledge on SV&V topics viz. requirements engineering, software reviews, configuration management, and software testing. Four key skill areas sought after by employers, namely communication skills, applied knowledge of methods, applied knowledge of tools, and research exposure are used to drive the development funded by a National Science Foundation grant and perfected through an industry-academia partnership. In this paper, we discuss in detail the four project plans the researchers and their industry counterparts followed over the past two years in the development and eventual dissemination of active learning tools. A course enhancement plan was used to drive activities related to reviewing, enhancing, and modularizing modules, identified by a gap analysis performed by focus groups comprised of industry and academic partners. The course delivery plan was used to drive activities related to developing content delivery strategies. An evaluation and assessment plan was used to drive activities related to periodically evaluating student learning and assessing the project. And finally a course dissemination plan is being used to drive activities related to disseminating course modules and assessment reports. Active learning tools have been disseminated through a workshop and other means to universities and industry partners. 1 Project Introduction With almost half of the world’s population (3.4 billion) relying on the internet, it has become nearly impossible to avoid the software footprint in everyday life [1] . Software is now ubiquitous and software has also become critical. In 2015, major software glitches affected numerous companies: Bloomberg (inoperable trading terminals), Royal Bank of Scotland (payments failure), Nissan (airbag sensory detectors malfunction), Starbucks (register malfunction) and the F 35 Joint Strike Fighter (targets detection failure). Each of these resulted in financial, brand, and more important bodily damages [2] . With mission critical and high-risk applications that have human lives and resources dependent on software applications, it is imperative to not only test for, but aim for zero defects. However, even after decades of development, the software industry continues to spend considerable time and resources dealing with the quality problem. In the US alone in 2007, the cost of failed software was estimated to be upwards of $75 billion in re-work costs and abandoned systems [3] . The fundamental challenge towards a solution that will improve software quality lies in the people and processes that develop and produce software. Acharya et al (2014) [4] reason that firstly, there is not enough awareness of the Software Verification & Validation (SV&V) benefits, and secondly, there are a lack of practitioners who understand the SV&V topics and processes adequately. Both the lack of awareness and personnel shortage considerably hinder significant progress in project success rates. Furthermore, Acharya et al [4] argue the root cause to be the lack of up-to-date SV&V courseware. To address this situation, a SV&V course curriculum has been improved at the author’s institution through a project funded by a National Science Foundation –Transforming Undergraduate Education in Science, Technology, Engineering, and Mathematics (NSF-TUES) grant. The goal of this project is to enhance and transform a SV&V course by incorporating academic research and industry best practices through an academia-industry partnership. This project achieved the following objectives: 1. Critically examined the existing SV&V course contents, 2. Identified areas where improvements could be made in pedagogy, 3. Developed 42 delivery hours of active learning tools, 4. Developed course delivery strategies, 5.Integrated and delivered new pedagogical tools in the course, 6. Performed assessments and evaluations of the effectiveness of these tools, and 7. Disseminated course modules and assessment reports. At this time dissemination to more institutions is being carried out. This project, scheduled for completion in August 2016, targets both undergraduate students and software practitioners and: 1. Improves SV&V knowledge and skills of students & practitioners, 2. Helps evolve a SV&V community, and 3. Improves SV&V teaching and learning opportunities. The end result is expected to improve product and process quality levels in the software development community, resulting in a larger and more skilled SV&V user community. This paper introduces this project, describes the academia-industry partnership, and provides an in-depth description of the project’s execution and outcomes. 2 Project Partners Academia-industry partnership is the key to achievement of the project outcomes. From the conceptual stage, it was believed that by working hand in hand with the industry, both the academia and industry would benefit. Students would gain knowledge that is currently practiced in the industry and which they would likely use in their professional career, and industry would be able to train/retrain their personnel using a tested delivery strategy. 2.1 Industry Partners – Formation and Role Understanding project requirements, studying possible partnerships, and selling the project vision was instrumental in getting the industry on-board. Three industry partners—Eaton Electrical Corporation, ServiceLink, and JDA Software Group— have supported this project from the time it was conceived. Additional partners, PNC Bank and ANSYS, were invited as this project progressed and the need for diversification was realized. These companies are either large software companies or companies with large software development activities in the areas of mortgage, intelligent pricing and revenue management, electrical meters, and engineering simulation. The project team is open to more industry joining the partnership but is selective in the industry domain. The role of the industry partner is well defined and involves the following: Critically reviewing and identifying knowledge gaps in SV&V courseware Assisting in developing course modules Delivering expert lecture sessions to undergraduate students at partner universities when requested Delivering training programs to industry practitioners Assessing student learning 2.2 Academic Partners – Formation and Role The author’s institute is collaborating with two categories of academic partners: Development Academic Partners and Implementation Academic Partners. These academic partners offer one or more bachelor degrees in the following areas: Software Engineering, Computer Science, Computer Engineering, and Electrical Engineering. These partners also share a strong desire to strengthen their programs. 2.2.1 Development Academic Partner and Activities Distinguished faculty members from the Milwaukee School of Engineering and Virginia State University (a HBCU partner) have supported this project from the very beginning as Development Academic Partners. Mutual interest is instrumental in this longstanding partnership. The role of the academic development partner is well defined and involves the following: Identifying at least one local industry partner involved in software development activities Working with assigned focus groups to critically review current course Developing six hours of course modules to address identified gaps in a content area familiar to the university program and its local industry partner(s) Assessing course contents through at least two delivery cycles 2.2.2 Implementation Academic Partners and Activities This project calls for delivery of the active learning tools, as well as dissemination of the tools and assessment reports. Six domestic and one international partner have supported this project including: Embry-Riddle Aeronautical University, Montana Tech, University of Michigan, Virginia State University, Fairfield University, Milwaukee School of Engineering, and ORT Braude College, Israel. Partners were identified through networking at the American Society for Engineering Education conferences, sharing of the author’s research. A workshop held in the author’s institute in August 2015 led to the inclusion of five other implementation academic partners namely: Auburn University, East Carolina University, Kennesaw State University, Bowie State University, and Clarion University. The role of the academic implementation partner is well defined and involves the following: Using entire or partial courseware developed by this project in at least one course through at least two delivery cycles, and Evaluating the course(s) and assessing the instruction at several levels. In addition, at the request of respective faculty members, the active learning tools have been shared with University of Alaska Southeast, Georgia Southern University, University of South Carolina–Upstate, Indiana University Southeast, Rose-Hulman Institute of Technology, State University of New York College at Oneonta, and Eastern Mediterranean University (Cyprus). 2.2.3 Focus Groups To ensure project plans were adequately executed, focus groups comprised of PI and/or co-PIs and academic development partners and industry partners were formed based on individual’s interest and expertise in the projects four SV&V topic areas: requirements engineering, software reviews, configuration management, and software testing. Well-defined