Premium
EVALUATING AND PREDICTING SURVEY EFFICIENCY USING GENERALIZABILITY THEORY
Author(s) -
JOHNSON SANDRA,
BELL JOHN F.
Publication year - 1985
Publication title -
journal of educational measurement
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.917
H-Index - 47
eISSN - 1745-3984
pISSN - 0022-0655
DOI - 10.1111/j.1745-3984.1985.tb01051.x
Subject(s) - generalizability theory , interpretability , consistency (knowledge bases) , computer science , conceptualization , sample (material) , sampling (signal processing) , test theory , appeal , process (computing) , domain (mathematical analysis) , psychology , management science , statistics , artificial intelligence , psychometrics , mathematics , engineering , political science , chemistry , filter (signal processing) , chromatography , law , computer vision , operating system , mathematical analysis
A long‐term science performance monitoring program began in England, Wales, and Northern Ireland in 1980 with the first in an initial series of annual national sample surveys of the science performances of II‐, 13‐ and 15‐year‐old pupils. The assessment framework underlying this program is process‐oriented, consisting of a number of subcategories of science activity, some of which are assessed in practical mode. Pupils are randomly selected for testing according to a complex sampling scheme. Questions are also selected randomly to represent the various subcategories. From the start of this program, it was intended to appeal to generalizability theory for a suitable estimation paradigm, and in this paper some preliminary applications of G‐theory are described. The results of these applications would suggest that computerized question‐banking, domain‐sampling of questions, and G‐theory together provide a useful new technology for this kind of performance monitoring exercise. The issue of interpretability might still remain a problem, however, unless the question domains can be clearly defined, and can be reflected in the question pools with consistency over time.