Open AccessRELIABILITY OF THE TEST OF SPOKEN ENGLISH REVISITED
Author(s) -
Boldt R. F.
Publication year - 1992
Publication title -
ets research report series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.235
H-Index - 5
ISSN - 2330-8516
DOI - 10.1002/j.2333-8504.1992.tb01483.x
Subject(s) - reliability (semiconductor) , psychology , inter rater reliability , scale (ratio) , variance (accounting) , test (biology) , section (typography) , consistency (knowledge bases) , internal consistency , statistics , psychometrics , applied psychology , rating scale , computer science , clinical psychology , mathematics , developmental psychology , artificial intelligence , geography , paleontology , power (physics) , physics , cartography , accounting , quantum mechanics , business , biology , operating system
The Test of Spoken English (TSE) is an internationally administered instrument for assessing nonnative speakers' proficiency in speaking English. The research foundation of the TSE ® examination is currently described in the Manual for Score Users. This publication describes technical characteristics of the test, including such psychometric characteristics as level of difficulty, reliability, and validity. Consistent with the ETS Standards for Quality and Fairness , the Manual refers to two sources of variation other than the achievement being measured: interrater reliability and internal consistency. Because the reported data on both issues were drawn from a 1980 study of reliabilities based on two raters, newer data and more extensive analyses were needed. The present study uses data from recent administrations of the TSE examination. Analysis of variance examined the effects of scale, section, examinee, and rater, as well as the interactions of these factors. Reliabilities were reported for item, section, and scale scores. Common and unique variance estimates were developed for each scale score for which a section was rated. Estimates of the effects of altering section lengths suggested that some sections should be lengthened and others omitted if reliability were to be maximized. Other suggestions were offered for improving reliability.