Open AccessA TREE‐BASED ANALYSIS OF ITEMS FROM AN ASSESSMENT OF BASIC MATHEMATICS SKILLS
Author(s) -
Sheehan Kathleen,
Mislevy Robert J.
Publication year - 1994
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.1994.tb01587.x
Subject(s) - paragraph , citation , mathematics education , psychology , library science , computer science , artificial intelligence , mathematics , world wide web
The operating characteristics of 114 mathematics pretest items from the Praxis I: Computer Based Test were analyzed in terms of item attributes and test developers' judgments of item difficulty. Item operating characteristics ere defined as the difficulty, discrimination, and asymptote parameters of a three parameter logistic item response theory (IRT) model. Three types of item attributes were considered: (1) surface features (for example, whether or not the item stem contained an equation); (2) aspects of the solution process (for example, whether or not the solution required application of a standard formula); and (3) response type (free-response or multiple-choice). Because the attribute set included large numbers of categorical variables, an approach based on binary regression trees (Breiman, Friedman, Olshen, and Stone, 1984) was implemented. The results were quite impressive for asymptote parameters (85% of variance explained), somewhat less so for difficulty parameters (36% of variance explained) and fairly unimpressive for discrimination parameters (only 12% of variance explained). In addition, the tree-based approach was found to be particularly useful for identifying important interaction effects and for developing graphical summaries of the modeling results. Six tables and eight figures support the analyses. (Contains 11 references.) (Author/SLD)