A Comparison of Peer Evaluation Methods in Capstone Design

The final chemical engineering capstone design course at our university consists of a semesterlong project where students work in predetermined groups. The emphasis of the experience is to allow students to develop and explore their engineering creativity while at the same time cultivate the importance of communication and teamwork skills. Students receive an industrially relevant open-ended problem statement and face a rigorous schedule with specific weekly deliverables. We concentrate on building teamwork skills, ownership of the project, accountability, communication and self-assessment. We use the projects to incorporate the skills our graduates need to join the workforce or graduate schools while inspiring students to evaluate, defend, and treasure their creations. Measuring group dynamics, division of labor, engagement, group versus individual experiences is a challenging task. Student grades are highly dependent on peer evaluation, as the team does not receive the same grade for group assignments, but instead, group grades are multiplied by a peer assessment factor (for example if student’s X peer assessment is 50 percent, all group grades for that student will be multiplied by 0.5. In the spring of 2012, two separate methods were employed as part of peer assessment: a traditional “point division” method 1 and CATME, the Comprehensive Assessment of Team Member Effectiveness 2 . The point division method instructs students to divide 100 points between the group members; if there are five team members in a group and a student believes all the students in the group participated equivalently in the project then he/she will assign 20 points to every student in the group. CATME allows the students to evaluate team members in more behavioral areas: contribution to the team, interaction with peers, keeping team on track, quality of work, proficiency in the discipline. Because the point division method is effectively a zero-sum game whose philosophy may contradict the cooperative nature of the course, we are interested in replacing it, and therefore have explored correlations between it and CATME. Additionally, we used the data from both methods to evaluate how students rated themselves compared to the ratings of their peers. In this report we share the results of our analysis. We have determined that the two methods have a statistically significant positive correlation and that, with modest significance, students on average self-evaluate higher than their peers evaluate them.