Open AccessRELIABILITY OF ADVANCED PLACEMENT EXAMINATIONS
Author(s) -
Bridgeman Brent,
Morgan Rick,
Wang Mingmei
Publication year - 1996
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.1996.tb01681.x
Subject(s) - reliability (semiconductor) , inter rater reliability , statistics , computer science , variation (astronomy) , psychology , reliability engineering , mathematics , rating scale , engineering , power (physics) , physics , quantum mechanics , astrophysics
The various methods for computing the reliability of scores on Advanced Placement examinations are summarized. For the free response portion of the examinations, raters can contribute to score unreliability through both systematic severity errors (in which some raters consistently rate more severely than other raters) and through inconsistency. Inconsistency appears to be a much greater problem then systematic severity errors. Question to question variation (or score reliability) is seen as a greater problem than rater inconsistencies. The impact of increasing or decreasing the number of topics is demonstrated by showing the proportion of students correctly classified as the number of topics changes. Procedures to enhance both rater and score reliability are discussed.