Premium
A PROBABILITY‐BASED FORMULA FOR CALCULATING INTEROBSERVER AGREEMENT
Author(s) -
Yelton Ann R.,
Wildman Beth G.,
Erickson Marilyn T.
Publication year - 1977
Publication title -
journal of applied behavior analysis
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.1
H-Index - 76
eISSN - 1938-3703
pISSN - 0021-8855
DOI - 10.1901/jaba.1977.10-127
Subject(s) - observer (physics) , statistics , agreement , psychology , interval data , inter rater reliability , interval (graph theory) , mathematics , econometrics , physics , combinatorics , quantum mechanics , data envelopment analysis , philosophy , linguistics , rating scale
Estimates of observer agreement are necessary to assess the acceptability of interval data. A common method for assessing observer agreement, per cent agreement, includes several major weaknesses and varies as a function of the frequency of behavior recorded and the inclusion or exclusion of agreements on nonoccurrences. Also, agreements that might be expected to occur by chance are not taken into account. An alternative method for assessing observer agreement that determines the exact probability that the obtained number of agreements or better would have occurred by chance is presented and explained. Agreements on both occurrences and nonoccurrences of behavior are considered in the calculation of this probability.