Demographic Effects on Student-Reported Satisfaction with Teams and Teammates in a First-Year, Team-Based, Problem-Based Course

This work-in-progress reports the effect of student gender and team gender make-up on team satisfaction and student assessments of team contributions. In first year team-based student design courses, instructors use student selfand peer-assessment information to gauge team functioning and even to affect student project scores. However, students’ identity characteristics, such as their gender and race, may impact the scores they receive from others as well as the scores they assign. The poster will also describe the creation of and results from a learninganalytics style investigation of the researcher’s own student team assessment data, and the poster presentation will allow others to query the data set with their own questions. The final data set includes assessment information from 620 first-year engineering students working in 132 teams of 4 or 5 to design, build, test, and report in varying open-ended project contexts (from 11 sections of the course taught between Fall 2011 and Winter 2015). The study is presented as a poster because the researcher is eager to connect with others at ASEE to consider new questions of interest. The data set includes demographic data (gender, race, TOEFL score when applicable, SAT/ACT scores, first year GPA, final course grade), as well as team assessments (student selfassessments and assessments of teammates, team report scores, and team satisfaction ratings). In this large data set, many comparisons were significant. Findings of interest regarding team satisfaction included a pattern of satisfaction by team gender breakdown. Teams with two or more women were much less happy (mean satisfaction of 3.79 on a 5-pt scale) compared to students on teams of just men (4.25) and teams with one woman (4.32). This finding is in contrast to a recommendation common in the literature to avoid stranding women on teams. Teams that included international students were less happy (mean satisfaction of 3.68) compared to teams that did not (4.24). Both of these patterns had medium-sized effect sizes (Cohen’s d was 0.67 for the finding by gender and 0.74 for the finding with international students.) There was no difference in the mean satisfaction of students on teams including or not including underrepresented minority students (4.16 for teams with URM students, 4.17 for teams without). Student ratings of peers were examined by gender. CATME asks students to rate each other on five scales for behaviors related to teamwork. The largest effect was found for “having related knowledge, skills, and abilities.” Women were rated lower on this category (by both men and women), and men rated women lower on this category than women rated women (Cohen’s d was 0.69 here). Women were rated higher than men on all other categories (by both men and women), but with much smaller effect sizes (Cohen’s d ranging from 0.12 to 0.32). This project is proposed as a work-in-progress because there are many limitations related to using my own data for this study. This exploratory study of existing data has uncovered interesting patterns in students’ assessments of team satisfaction and contributions that should be investigated further, perhaps through follow-up interviews or through an experimental design. I look forward to talking with conference attendees and getting input before moving forward with such a study. I am also eager to have conversations about what the findings might suggest for how teams are formed and, more importantly, how they are supported and assessed. Introduction and Motivation Engineering classes make use of team-based, project-based pedagogies throughout the curriculum, and such projects are particularly common in first year courses. Learning goals for such projects include teamwork, design, and communication. Instructors using this pedagogy want the project to go smoothly and efficiently, with teams working well together and all students contributing their fair share, a concept known in the psychology literature as “team effectiveness.” We don’t always know how to make effective teams happen, though. Instructors work to increase team effectiveness in their team-based, problem-based design courses using many pedagogies. One strategy that is the focus of many studies is on how teams are assigned. Suggestions common in the literature, which sometimes are at odds, include making heterogeneous teams (by gender, by skill level, by race), making homogenous teams (by gender, by skill level, by goals for the project), and avoiding stranding minority students (gender and race, specifically). Another concern of instructors is social loafing; we don’t want students to shirk their project responsibilities yet earn the same scores as their hard-working peers. Social loafing is a particular problem in first year projects, when each students’ contribution is generally not unique because the students typically don’t have the backgrounds to allow for interdisciplinarity. The tasks are not differentiated for students because they are all assumed to have the same set of skills. It is hard to measure effective teamwork, but instructors adopt many different methods of assessment to try to capture this information. Many of these methods rely on student assessments of their own and peer contributions. In general, when peer assessments might identify social loafers, social loafing decreases or even disappears. In an attempt to preempt social loafing, instructors conduct student assessments and sometimes even use this information to affect student grades. I am guilty of such a practice in my classrooms. However, studies of student selfand peerassessments often find systematic overvaluing of one’s own work and systematic undervaluing of the documentation steps in an engineering project. More importantly, the use of such assessments with real-world (grade) consequences also makes it critical that we consider whether students’ identity characteristics are affecting their self-reports or assessments of and from teammates. There is precedent to believe that self-assessments might be affected by student identity characteristics such as gender. Women in first-year engineering courses report lower engineering self-efficacy on a variety of instruments, but it is not known whether such responses will transfer to a team assessment report like is conducted in team-based learning contexts. Some studies have found gender effects in team assessment (and team grades), including that women on engineering teams are more critical in their assessments of other women. Okudan and colleagues also conclude that homogeneous teams (by gender) earn higher project scores than heterogeneous teams, but the result they report is not statistically significant. Using data similar to the data reported here, Van Tyne, Van Tyne, and Van Tyne investigated actual student team assessments (self and peer) from students who completed a first-year engineering design course. They found no significant differences by gender in either selfassessment or peer-awarded assessment, and attributed this finding to the support available at a small school focused only on engineering. This study adds to this result with a comparison group at a large public institution with a large proportion of non-engineering students.