Open AccessINSIDE SOURCEFINDER: PREDICTING THE ACCEPTABILITY STATUS OF CANDIDATE READING‐COMPREHENSION SOURCE DOCUMENTS
Author(s) -
Sheehan Kathleen M.,
Kostin Irene,
Futagi Yoko,
Hemat Ramin,
Zuckerman Daniel
Publication year - 2006
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.2006.tb02030.x
Subject(s) - computer science , comprehension , test (biology) , reading comprehension , reading (process) , quality (philosophy) , natural language processing , information retrieval , artificial intelligence , machine learning , data mining , philosophy , programming language , paleontology , epistemology , political science , law , biology
This paper describes the development, implementation, and evaluation of an automated system for predicting the acceptability status of candidate reading‐comprehension stimuli extracted from a database of journal and magazine articles. The system uses a combination of classification and regression techniques to predict the probability that a given document will be deemed acceptable for use in completing a specified passage‐creation assignment by at least one test developer. The text features that form the basis of the estimated models are automatically extracted by natural language processing techniques. Model performance is evaluated by comparing the proportion of acceptable documents located with the screening capability turned on to the proportion of acceptable documents located with the screening capability turned off. The evaluation suggests that the estimated models have succeeded in capturing useful information about the characteristics of texts that affect test developers' ratings of source acceptability and that they can help test developers find a greater number of high‐quality sources in less time.