Effects of Technology Modes on Ratings of Learner Recordings

While research has investigated the effect of visuals in tests of listeningcomprehension (e.g., Suvorov, 2009; Wagner, 2008, 2010), studentrecordedvideo for oral formative assessment is relatively unexplored.In this study, I examined 15 teachers’ ratings of speech recorded by 39ESL learners to see if teachers assess speech differently depending onwhether it is presented with visuals. The learners recorded 4 speechsamples: 2 with webcams, 2 with microphones only. A third speechcondition was created by removing the video track from the webcamrecordings, resulting in 3 conditions and 6 samples for each individual.The teachers rated all 6 samples. I used repeated-measures ANOVAs todetermine whether the teachers assigned significantly different scoresbased on the speech conditions. The results showed that the teachersrated the audio stripped from the video significantly higher than thevideo/audio recordings (p = .004, d = .38). This suggests that teachersmay be biased in favor of audio-only recordings and that teachersshould not give students an option of making either an audio or videorecording for a given formative assessment. Further analyses examinedhow the students’ and teachers’ preferences for audio-only or videorecordings were related to the ratings.