Refinement And Initial Testing Of An Engineering Student Presentation Scoring System

We have previously created and beta tested a workforce-relevant, research-based scoring system for use with engineering student presentations across multiple contexts. Since then, we have systematically validated, refined, and tested the rubric in a five-step process described in some detail for this paper. First, we tested the face validity and usability of the instrument via the collection of additional feedback during focus groups and interviews with: faculty possessing expertise in scoring system design, faculty with experience in engineering design projects that involve student presentations, and additional faculty from a variety of backgrounds. Second, we used this feedback to reduce overlap and complexity in the scoring system items. Third, teaching assistants and the researchers used the scoring system items to provide feedback to approximately 140 students on presentations in a senior design course. Fourth, we made additional modifications and simplifications to the system based on the insights gained from the TA feedback process. Fifth and finally, three raters applied the resulting scoring system to several videotaped student presentations to check for inter-rater reliability and evidence of construct validity. Based on the methodology above, we reduced the instrument from 36 items to 19 items. These items include using concrete examples and details familiar to the audience; consistently referring to how key points fit into the big picture; using graphics which are visually appealing, easy-to-understand, and include helpful labeling; and effectively combining energy, inflection, eye contact and movement; among others. This paper includes a description of the process used to create the instrument, a description of the instrument, the supplemental teaching guidelines under development, and a discussion of the materials’ potential for use across many engineering contexts.