A Framework For Assessing The Influence Of Variations Between Individual Capstone Experiences

Within the boundaries of a single capstone course, the experiences of different students can vary greatly. Different students tackle different problems, work with different clients, are advised by different faculty and are organized into unique teams. Such a range of experiences within a single course is not typical; many courses are designed with the opposing goal of providing as uniform of an experience as possible to all students. Due to this fundamental difference, that some courses aim to provide a uniform experience for all students while others embrace a range of experiences for individual students, the same evaluation framework cannot be applied to both types of courses. While an evaluation framework for a course with minimal variation between student experiences can assume that each student receives the same "treatment," the effect of differences between individual student experiences must be integrated into an evaluation framework for a capstone course within which there is a large range of individual experiences. In this paper, an assessment framework is introduced which assesses the effect that differences between capstone experiences have on student outcomes. The proposed framework identifies sources of variation in a student’s capstone experience that are endogenous to the course (e.g., project type, team size) and exogenous to it (e.g., students’ prior knowledge, students’ demographics). This approach emphasizes an understanding of the underlying aspects of each student’s individual capstone experience. In addition, differences that students bring with them to capstone, including prior industrial experience, academic performance, and demographic differences, are integrated into the framework. Understanding the connection between these differences and the fulfillment of desired outcomes is important to informing the structure of a capstone program. Through a detailed case study with Systems & Information Engineering capstone students at the University of East, it is established that variation between individual student capstone experiences influences student outcomes. Variation in gender, previous work experience and differences in the level of interaction with clients all influence the achievement of several measured objectives. The assessment framework reveals the connections between specific independent variables and outcomes. Strengths and weaknesses of the framework are discussed and opportunities for future work will be explored. Motivation for the Framework In response to critiques that engineering graduates lacked practical design skills relevant to industry, capstone design courses have become, during the last two decades, a standard component in the final year of most undergraduate engineering curricula 3, 2 . While each capstone course is unique, most include teams of students under the direction of a project P ge 1.28.2 advisor working on a real (or at least "realistic") design project 5 . Teams typically submit written and oral deliverables and may also implement and test an actual design. Processfocused capstone courses can provide a stark contrast to many outcome-focused engineering courses earlier in a curriculum in which the focus is more heavily weighted towards theoretical analysis tools. Another way in which capstone courses are different than engineering analysis courses is the variability between the experiences of individual students within a course. In an effort to provide a fair course for all students, many engineering analysis courses are designed to provide a unified experience to all students. All students attend the same classroom experiences (e.g., lectures), use the same textbook, have the same instructor, complete the same assignments, and take the same tests. While there are clearly examples where such a unified experience does not exist -for example, multi-section courses with different instructors -many of these exceptions are the result of practical logistics (e.g., classroom size) and faculty frequently strive to minimize the differences through using the same text, assigning similar homework, and using common examinations across multiple sections. A singular experience for all students makes sense from a workload perspective, too: a unique experience for each student would seem to require more time and energy from an instructor. This is not to say that a course designed to provide a uniform student experience does not treat each learner as a unique person with unique learning styles and pre-existing knowledge structures. To the contrary, “uniform experience courses” can be designed to be “robust to” differences among students. As engineers are known for designing systems to be robust to differences between individual users and other "noise" factors, it would be only fitting that engineers would design courses to be robust to differences between individual students. Whereas the aim in many outcome-focused engineering analysis courses is to minimize the variability between the experiences of individual students, such variability is frequently integrated into process-oriented capstone courses. Students work on different teams, each working on different projects or different parts of the same larger project. A single person does not always advise all of the capstone teams and deliverables for each team may not be the same. Even when deliverables are identical between teams, expectations for those deliverables among the different project advisors can vary. Some teams may have industrial clients, others may be working on “design contest” projects (e.g., SAE Mini-Baja, Solar Car), others may be working for faculty as part of the faculty member's research, while others may be working on projects of their own creation. Students may be working on teams composed of students from several majors or they may be working on teams with students only from their own major. The variability between individual capstone experiences is rooted, at least in part, in necessity. For large programs, it is simply impossible to provide a realistic design experience that is uniform for all students. The variability, however, can also be designed into a program purposefully. Such purposeful variability can allow students to pursue an experience that best meets their needs. Which project topics, project advisors, client types, etc., would help the student meet their objective most effectively?