Premium
Agreement between observers when the categories are not specified in advance
Author(s) -
Brook Richard J.,
Stirling W. Douglas
Publication year - 1984
Publication title -
british journal of mathematical and statistical psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.157
H-Index - 51
eISSN - 2044-8317
pISSN - 0007-1102
DOI - 10.1111/j.2044-8317.1984.tb00805.x
Subject(s) - mathematics , categorization , test (biology) , variety (cybernetics) , test statistic , statistic , type (biology) , distribution (mathematics) , statistics , statistical hypothesis testing , computer science , mathematical analysis , artificial intelligence , paleontology , ecology , biology
To test the agreement between two observers who categorize a number of objects when the categories have not been specified in advance, Brennan & Light (1974) developed a statistic A ′ and suggested a normal approximation for its distribution. In this paper it is shown that this approximation is inadequate particularly when one, or both, of the observers place a fairly equal number of objects in all of their categories. A chi‐squared approximation to the distribution of A ′ is developed and is shown to work well in a variety of situations. The relative powers of A ′ and the ordinary X 2 test for association are dependent on the type of ‘agreement between the observers’ that is assumed. However a simulation for a fairly general type of agreement indicates that the X 2 test is more powerful. As the X 2 test is also much easier to apply, it would seem preferable in most situations.