Premium
A CATEGORICAL INSTRUMENT FOR SCORING SECOND LANGUAGE WRITING SKILLS
Author(s) -
Brown James Dean,
Bailey Kathleen M.
Publication year - 1984
Publication title -
language learning
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.882
H-Index - 103
eISSN - 1467-9922
pISSN - 0023-8333
DOI - 10.1111/j.1467-1770.1984.tb00350.x
Subject(s) - generalizability theory , psychology , categorical variable , grammar , reliability (semiconductor) , inter rater reliability , writing assessment , mathematics education , linguistics , rating scale , computer science , developmental psychology , power (physics) , physics , quantum mechanics , machine learning , philosophy
This paper first reviews recent research on evaluating second language learners’writing skills. It then discusses research on a categorical instrument for evaluating compositions written by upper intermediate university ESL students. The form of the instrument used in this study included five equally weighted criteria for scoring: (1) Organization, (2) Logical Development of Ideas, (3) Grammar, (4) Mechanics, and (5) Style. An experiment was conducted under controlled conditions in which ten raters scored fifty randomly selected compositions. Regression analysis and generalizability theory were used for investigating the reliability of the instrument. In addition, information was obtained from the raters as to their reactions to the instrument. The results indicate that the scoring instrument is moderately reliable. More useful, perhaps, is the demonstration provided of the effects on reliability of changing the number of raters and/or criteria. The paper concludes with a discussion of the issues involved (both with this particular instrument and with analytic scoring systems in general) and of possible future research on the evaluation of nonnative speakers’writing.