Evaluating assessment in a School of Economics

Previous research undertaken by one of the authors identified a general concern among undergraduate students in large business courses associated with the large number of sessional staff. In particular, students expressed their view in the focus groups with staff their concern that a large number of markers may affect their performance in essay style examinations as a result of the inevitable variation in severity between different raters. The aim of this paper is to investigate whether these concerns are justified. The focus of this study was the weekly tutorial papers that were submitted for marking by 400 students enrolled in a first year Principles of Microeconomics course. These papers were marked by a team of ten markers whose experience in university teaching ranged from four weeks to over 30 years. 40 per cent of these papers were triple marked by two other raters in order to fully separate the student by rater by item interactions during the subsequent statistical analysis. The results obtained from this triple marking exercise were then analysed using ConQuest 2.0, which uses logistic regression to provide estimates of the parameters of the Partial Credit Model. The Partial Credit Model measures variations in rater severity and four other common rater errors, the halo effect, the central tendency effect, the restriction of range effect and the inter-rater variability or consistency. The study identified the presence, to some degree or other, of all five rater errors, even among the most experienced raters. The paper concludes by suggesting that the key to