Assessing Team Effectiveness: Comparing Peer Evaluations To A Team Effectiveness Instrument

Engineering education continually increases the use of academic teams for active and cooperative learning. According to ABET, engineering students should be able to function effectively within a multidisciplinary team. Moreover, current engineering practice in industry requires effective team cooperation. Thus being able to function in a multidisciplinary team entails both academic and industrial importance. Team effectiveness has been studied extensively in industrial settings. Many factors that contribute to the team effectiveness have been studied thoroughly. Factors including interdependency, potency, and goal setting have been identified as highly related with team effectiveness. This paper addresses the validity of our team effectiveness scale through cross-validation process. In order to do so, we developed a 9-item Likert self-assessment peer evaluation scale to measure how individual evaluate their peer teammates as effective or not. Psychometric analysis results are used to demonstrate reliability of the data sets. Cronbach’s coefficient alpha is higher than .90 for both the peer evaluation scale and the team effectiveness scale. Confirmatory Factor Analysis (CFA) is used to verify the theoretical structure of the peer evaluation and team effectiveness factors using LISREL. CFA results show there is a positive correlation between the team effectiveness measured by the two scales, thus we concluded that our team effectiveness instrument proved to be valid through the cross-validation process. Background The Accreditation Board for Engineering and Technology (ABET) [1] with Engineering Criteria 2000 started a movement to advance the current curriculum and pedagogy of engineering education. According to ABET guidelines, students graduating from engineering programs should not only have strong traditional engineering knowledge in fundamental areas such as mathematics and science, but should also be able to work effectively in a multidisciplinary environment in multicultural teams. Campion, Medsker, and Higgs [2] define team effectiveness in terms of productivity, employee and customer satisfaction and manager judgments. Based on this hypothesis, they found that potency and interdependency are among factors described as important attributes of an effective team through the study of real teams in the field. Guzzo [3] defines team effectiveness through group-produced outputs and the capability to perform well in the future. O’Leary-Kelly, et al. [10] proposed that goal setting has a strong effect on effective team performance through metaanalytic approach. After reviewing many laboratory and field studies on the effects of a task, Locke et al found that specific and challenging goals setting contributes better performance [9]. Imbrie et al. [4] operationalized team effectiveness through interdependency, potency, goal setting and learning. P ge 14249.2 Theoretical Model Imbrie [4] reported results of an analysis of a four-factor model to study team effectiveness of student teams in real academic setting. A 24-item Likert team effectiveness scale was developed to measure the following four factors: interdependency, potency, goal setting and learning. In Imbrie’s study, a confirmatory factor analysis results show the four factors model provides no better fit than a one-factor model. In this study, we look at three factors of the four-factor model interdependency, potency and goal setting, this corresponding to 19-item Likert team effectiveness scale. The analysis in this report is based on a three factor team effectiveness model. We have developed two instruments for this study. The first is a 19-item questionnaire instrument measuring a student’s perception about his/her team’s overall performance in the three factors of the team effectiveness, as shown in Table 1. The latent variable expressing the student’s perception of their team’s overall performance is designated as TETEAM. In Table 1, items starting with IN, GS and PT correspond to interdependency, goal setting and potency measurement respectively. All items are in a 5-item Likert-scale from Strongly Agree to Strongly Disagree. Table 1: Team effectiveness items for measuring a student’s general experience on how effective they found their team experience (designated as TETEAM) Items Description--Interdependency IN01 My team collaborated effectively to complete our assignments. IN10 My teammates displayed appropriate interpersonal skills when conflict arose. IN02 My contributions to the team were appreciated by each teammate IN03 I had confidence in each team member to contribute his/her fair share of what was required. IN04 My team used a process/method (e.g., code of cooperation) to hold each member accountable. IN09 Team members were prepared for team meetings. IN08 Team members arrived on time to team meetings. IN06 An outside observer would have concluded our team had an effective process to complete our assignments IN05 At any particular time, I knew what each member of my team's role was so I knew what to expect from them. PT01 My team was confident in its ability to overcome adversity (e.g., interpersonal conflict, assignments). PT02 I feel a sense of accomplishment in my team's ability to work together. PT03 This team gave me confidence in the ability of teamwork to solve problems. PT04 My team had the collective abilities (e.g., communication, interpersonal, technical) to accomplish course assignments. PT05 I was confident that our team produced acceptable solutions to course assignments. GS02 My team used clear, long term goals to complete tasks. GS03 My team reflected upon its goals in order to plan for future work. P ge 14249.3 The second questionnaire is a 10-item Likert-scale peer evaluation instrument developed to measure three team effectiveness factors based on how a student evaluates each individual on his/her team. Among the 10-item Likert-scale peer evaluation, there is one single item on its own dedicated to measure the general opinion of each specific team member on their whole team effectiveness is designated TECT. The team effectiveness measured by the 9-item is designated as TEPEER. The team effectiveness from the one general team effectiveness in the peer evaluation instrument is designated as TECT. The detailed list of the items is shown in Table 2. Constructs are labeled I, G and P, representing interdependency, goal setting and potency, shown as the last letter of “Item ID” in Table 2. Table 2: Peer evaluation items for measuring how a student evaluating their peers. 9-items within TEPEER: Item ID Item Description CI021I Collaborates well with my team on all in-class and out of the class assignments. CI022I Contributes to my team's effectiveness by having a clearly defined role(s). CI023I Is a reliable team member. CI024G Often helps my team think of what we were/were not achieving. CI025G Articulates individual goals that can be achieved with the help of my team. CI026G Actively helps my team establish goals. CI027P Helps my team to build a shared confidence in its ability to successfully work together on course assignments. CI028P Often encourages each team member to believe in my team's ability to succeed no matter what the task. CI029P Often makes my team feel confident in its ability to resolve disagreements. Single item designated TECT: CT1 Overall, I would consider my team to be highly effective. In this study we try to investigate the validity of our team effectiveness scale through a cross validation process. The research question of this study: is the team effectiveness scale valid through cross-validation process? Methods Participants of this study include 879 freshmen engineering students at a large Midwestern University. The population consists of students that are 22.6% female, 77.4% male. All students are enrolled in the same first year engineering course. Students were assigned to a permanent team consisting of 3 or 4 students through the entire semester working on course related projects. The team was formed by taking into account of student’s background diversity. The self/peer evaluation questionnaire (TEPEER, TECT) was given to students after finishing their first project as a team. The team effectiveness instrument questionnaire (TETEAM) was given to the students immediately after they completed the self/peer evaluation. GS04 My team made use of incremental goals (i.e., we set short term goals) in order to complete course assignments on time. GS05 My input was used to set our team goals. GS06 This team helped me accomplish my individual goals for this course.