Using Videos in ESL Listening Achievement Tests: Effects on Difficulty

The question as to whether an assessment construct of second language (L2) listening comprehension should include the decoding of visual information remains unanswered (see Buck, 2001; Ockey, 2007). This study aimed to fill this gap by investigating how audio-only and video-enhanced delivery formats of listening passages compared in terms of difficulty for English as a second language (ESL) students. It utilized students’ performance on listening achievement tests developed by the researcher. The participants were 60 low- and high-proficiency ESL students enrolled in an American intensive English program. The participants’ scores on the achievement tests were used to compare the difficulty of items of different formats and determine whether this difficulty related to video type (context versus content) and students’ proficiency level. The findings suggested that, at least for higher-level students, listening testlets enhanced with videos containing mostly content-related visuals were significantly easier than their audio-only counterparts were. On the contrary, the inclusion of videos with mostly context visuals did not affect the difficulty of testlets in any proficiency category. The findings are discussed in terms of their practical significance for ESL teachers as well as theoretical implications for the field of ESL listening assessment.