Open AccessKEYLIST ITEMS FOR THE MEASUREMENT OF VERBAL APTITUDE
Author(s) -
Ward William C.,
Dworkin Dan,
Carlson Sybil B.
Publication year - 1986
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2330-8516.1986.tb00183.x
Subject(s) - aptitude , psychology , context (archaeology) , meaning (existential) , test (biology) , set (abstract data type) , constraint (computer aided design) , construct (python library) , cognitive psychology , item bank , item response theory , mathematics education , social psychology , natural language processing , psychometrics , computer science , developmental psychology , mathematics , paleontology , geometry , psychotherapist , biology , programming language
Two verbal item types employed in standardized aptitude tests were administered in a conventional multiple‐choice format and in the keylist format, in which the examinee is required to think of an answer, look it up in a long ordered list, and write its number. The keylist format provides a machine‐scorable surrogate for a truly free‐response test. Its potential attractions include the increased acceptability of items given in a production rather than a recognition format, resistance to coaching based on “gaming” strategies for eliminating multiple‐choice alternatives, and elimination of the need in item writing to produce plausible distractors for an item. Relations among tests employing different response formats were analyzed and their correlations with other measures of aptitude and achievement were compared. As in several previous studies, these analyses indicated that the format has little or no systematic effect on the construct validity of tests employing item types used in standardized tests of verbal aptitude. One of the purposes of the study was to determine the degree to which experienced test developers could agree on the set of keys that should be supplied for each keylist item. Agreement among reviewers was far from the near‐perfect consensus that would be required for use of this format, perhaps because the two item types investigated, Antonyms and Analogies, represent tests dealing with word meanings taken largely out of context. Many English words can convey multiple shades of meaning and can be constrasted along multiple dimensions. Without the constraint imposed by context, the number of possibly acceptable answers can become unmanageably great, particularly if it is required that all acceptable keys be included in the list and that all that are included must be clearly defensible. Several suggestions were offered of situations in which variations on this format could appropriately be employed.