Open AccessELECTRONIC SOURCES AS INPUT TO GRE ® READING COMPREHENSION ITEM DEVELOPMENT: SOURCEFINDER PROTOTYPE EVALUATION
Author(s) -
Passonneau Rebecca,
Hemat Lisa,
Plante Jan,
Sheehan Kathleen M.
Publication year - 2002
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.2002.tb01879.x
Subject(s) - computer science , variety (cybernetics) , selection (genetic algorithm) , reading (process) , comprehension , source code , set (abstract data type) , process (computing) , test (biology) , reading comprehension , task (project management) , natural language processing , artificial intelligence , information retrieval , programming language , systems engineering , engineering , paleontology , political science , law , biology
This evaluation study compares the performance of a prototype tool called SourceFinder against the performance of highly trained human test developers. SourceFinder — a specialized search engine developed to locate source material for Graduate Record Examinations (GRE ® ) reading comprehension passages — employs a variety of shallow linguistic features to model the search criteria employed by expert test developers, to automate the source selection process, and to reduce source‐processing time. The current evaluation provides detailed information about the aspects of source variation that are not well modeled by the current prototype. Approaches for enhancing performance in identified areas are discussed. The present study also provides a more explicit description of the source selection task, and a rich data set for developing a less subjective, more explicit definition of the types of documents preferred by test developers.