Assessment of Project Completion for Capstone Design Projects

The Capstone Design project has become a significant educational experience for the student in the preparation of their professional skills and the application of their growing technical expertise. The typical design process experience includes Problem Definition, Concept Generation, Preliminary Design, Detail Design, and Communication of Results. However, the end result of this process is ultimately Solution of the Problem which often receives little attention in the academic setting. A successful Capstone Design program including companion design courses has been developed that has become an integral and important component of the Mechanical Engineering curriculum at the University of California, Santa Barbara. A variety of challenging projects are created each year to appeal to varied student academic and career interests. Students work in teams with the assistance of a faculty advisor to tackle a significant Mechanical Engineering Capstone Design project. The design experience and course experience includes defining the problems to be addressed with formal Design Requirements and identifying how the problems will be solved with a formal Project Plan. As the design and the project evolve, the course experience includes a Design Review and an Engineering Report. For those students that have not had the benefit of professional work experience or internships, these course deliverables provide an initiation and foundation for their professional engineering careers. Objective assessment of the course deliverables is difficult for Capstone Design projects and courses. There is excellent published literature that provides guidance based upon learning outcomes and the design process. The design process within the Capstone course may be conducted in three assessments including Problem Scoping, Concept Generation and Solution Realization. There is some published literature regarding scoring rubrics that are helpful for assessing communication skills as demonstrated in a report or presentation for course grading purposes. However, there is a noted absence regarding the expectations and assessments regarding the final outcome of the design project, Solution of the Defined Problems (or Solution Assets). Little attention has been given by instructors and as a result little guidance has been provided to the students regarding the expectations and assessment of Project Completion within the context of Solution Realization for Capstone projects. The Capstone project provides a unique experience regarding problem solving for the student. The students and teams should be held accountable in providing a formal definition of expected Project Completion outcomes and should provide objective evidence of problems solution and project completion. This paper will address the course deliverables and experiences of the demonstration and assessment of Project Completion as it has evolved. The evolution of the course deliverables now more clearly addresses Project Completion and Problem Solution. Students and teams have demonstrated the ability to define the problems that need to be solved and now must clearly provide objective evidence of solving the problems that have been defined. This assessment is conducted separate from the quality of communication skills typically assessed by a final report or design review. Introduction The Capstone Design Projects course is now required for all senior Mechanical Engineering students at the University of California, Santa Barbara and is taken in the final year for two units each quarter. This course was developed to provide the students an experience of working as a project team to address a practical and significant design and build project. The importance of individual roles and responsibilities while working as a team is emphasized. Students work in teams of three to five with the assistance of a faculty adviser. Engineering deliverables in the form of Prototyping, Testing, Modeling, and Analysis are required each quarter. Engineering communication in the form of oral presentations, design reviews, and reports is addressed throughout the course and project experience. We emphasize a practical, hands-on experience, and integrate analytical and design skills. The ME Capstone Design Projects include the following types of projects: 1) Industry Partner projects are supported by gifts to the program. Students have an opportunity to work on practical design projects and to interact with outside engineers. 2) Research Partner projects are supported by research or University funding to support current University research projects. Students have an opportunity to work with leading international researchers, graduate students, and research laboratories. 3) Student Organizations and Design Competitions include Engineers Without Borders and SAE design competitions. 4) Student and Faculty Created Projects include creative and challenging projects with an entrepreneurial perspective. A variety of challenging projects are created each year to appeal to varied student academic and career interests. Student project selection and team formation can be a challenging and time consuming process that is critical to the success of the design project and course experience. Successful student teams should include enthusiastic, motivated and engaged students. This paper will not address how that projects are selected as this has been previously reported. The typical design process experience includes Problem Definition, Concept Generation, Preliminary Design, Detail Design, and Communication of Results. However, the end result of this process is ultimately Solution of the Problem which often receives little attention in the academic setting. There is excellent published literature that provides guidance based upon learning outcomes and the design process. Common design phases, processes, and products for Capstone projects has been described and provided a foundation for the course experience. Classroom learning activities have also been described. There is some published literature regarding the review and assessment of design projects. However, there is a noted absence regarding the expectations and assessments regarding the final outcome of the design project, Solution of the Defined Problems (or Solution Assets). An attempt to assess design outcomes has been described in an attempt to measure the “goodness” of a design outcome. A Customer Satisfaction Survey and Design Quality Rubric were used. Although of limited use these instruments fail to capture and assess the Solution Assets with Solution of the Problem. The use of scoring rubrics for the assessment of senior design projects has been described. However, use of these rubrics are focused more upon “...streamlining the assessment process, resulting in a reduction in faculty assessment workload.” The assessment of the quality of student design reports using rubrics has been described. These rubrics, however, are limited to evaluating reports and indicated that refinement was necessary. An attempt to assess creativity in design assessment has been the focus of interesting research but does not yet provide practical assessment methodology. The use of Management By Objective (MBO) has been reported and provides good insight into the use of students establishing self goals and evaluation of goal achievement. The use of scoring rubrics that are helpful for assessing communication skills as demonstrated in a report or presentation for course grading purposes has also been reported. Literature of particular interest describes assessments for three performance areas in Capstone Engineering Design. The authors defined four areas of performance: 1) Personal Capacity 2) Team Processes 3) Solution Requirements 4) Solution Assets Assessment instruments have been developed for three of the four performance areas with Solution Assets remaining to be addressed. This paper is intended to address the development and evolution of the course deliverables as they relate to the design process, the course experience and Project Completion. Difficulties were encountered and addressed over the years regarding the assessment of the final outcomes of the design projects. The evolution of the course deliverables now more clearly addresses Project Completion and Problem Solution. The Capstone design project provides a unique experience regarding problem solving for the student. This paper will provide examples of course deliverables and provide some guidance for other programs to consider. Capstone Design Projects Course Outcomes and Objectives The Capstone Design Projects course is typical for a course of this type. The topics covered by the course include: Formation of Teams Development of a Project Plan Design Research Development of Concepts and Designs Design Development Prototyping Design Analyses Testing and Evaluation Final Design Engineering reporting including presentations, design reviews, and technical reports The course grading was originally in-progress grading for the Fall and Winter quarter. A final grade was determined in the Spring quarter after completion of the project and all course deliverables. After determination of the final grade, this was reflected retroactively for the Fall and Winter quarters. This created administrative issues and created an atmosphere of delayed achievement with a negative impact upon the course. In-progress grading was abandoned after the second year of experience in favor of well defined course deliverables with team grades and individual grades determined each quarter. The development of standardized course deliverables for each academic quarter with a variety of challenging projects was required in the academic setting. Common design phases, processes, and products for Capstone projects has been described and provided a foundation for the course experience and the development of course deliverables (see Table 1 and Table 2). Table 1. Comm