Open AccessEFFECT OF IMMEDIATE FEEDBACK AND REVISION ON PSYCHOMETRIC PROPERTIES OF OPEN‐ENDED GRE ® SUBJECT TEST ITEMS
Author(s) -
Attali Yigal,
Powers Don
Publication year - 2008
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.2008.tb02107.x
Subject(s) - subject (documents) , test (biology) , psychology , reliability (semiconductor) , correctness , natural language processing , mathematics education , cognitive psychology , computer science , world wide web , paleontology , power (physics) , physics , quantum mechanics , biology , programming language
Registered examinees for the GRE ® Subject Tests in Biology and Psychology participated in a Web‐based experiment where they answered open‐ended questions that required a short answer of 1‐3 sentences. Responses were automatically scored by natural language processing methods (the c‐rater ™ scoring engine) immediately after participants submitted their responses. Based on natural language processing methods (the c‐rater scoring engine), participants received immediate feedback on the correctness of their answers and an opportunity to revise their answers. A significant revision effect was found. Participants were able to correct many of their initial incorrect answers, resulting in higher revised scores. In addition, the reliability of revised scores was higher than initial scores, although the correlations of the initial and revised scores with the GRE Subject Test scores were similar.