Premium
Marking essays on screen: An investigation into the reliability of marking extended subjective texts
Author(s) -
Johnson Martin,
Nádas Rita,
Bell John F.
Publication year - 2010
Publication title -
british journal of educational technology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.79
H-Index - 95
eISSN - 1467-8535
pISSN - 0007-1013
DOI - 10.1111/j.1467-8535.2009.00979.x
Subject(s) - reliability (semiconductor) , scripting language , consistency (knowledge bases) , variety (cybernetics) , mode (computer interface) , set (abstract data type) , psychology , focus (optics) , affect (linguistics) , computer science , gold standard (test) , linguistics , artificial intelligence , statistics , communication , human–computer interaction , mathematics , power (physics) , quantum mechanics , physics , philosophy , optics , programming language , operating system
There is a growing body of research literature that considers how the mode of assessment, either computer‐based or paper‐based, might affect candidates' performances. Despite this, there is a fairly narrow literature that shifts the focus of attention to those making assessment judgements and which considers issues of assessor consistency when dealing with extended textual answers in different modes. This research project explored whether the mode in which a set of extended essay texts were accessed and read systematically influenced the assessment judgements made about them. During the project, 12 experienced English literature assessors marked two matched samples of 90 essay exam scripts on screen and on paper. A variety of statistical methods were used to compare the reliability of the essay marks given by the assessors across modes. It was found that mode did not present a systematic influence on marking reliability. The analyses also compared examiners' marks with a gold standard mark for each essay and found no shifts in the location of the standard of recognised attainment across modes.