Premium
MULTIPLE RATERS IN SURVEY‐BASED OPERATIONS MANAGEMENT RESEARCH: A REVIEW AND TUTORIAL
Author(s) -
Boyer Kenneth K.,
Verma Rohit
Publication year - 2000
Publication title -
production and operations management
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.279
H-Index - 110
eISSN - 1937-5956
pISSN - 1059-1478
DOI - 10.1111/j.1937-5956.2000.tb00329.x
Subject(s) - terminology , computer science , set (abstract data type) , reliability (semiconductor) , quality (philosophy) , core (optical fiber) , exploratory research , field (mathematics) , management science , data science , operations research , mathematics , telecommunications , philosophy , linguistics , power (physics) , physics , epistemology , quantum mechanics , sociology , anthropology , pure mathematics , economics , programming language
Research in the area of operations strategy has made significant progress during the past decade in terms of quantity of articles published, as well as the quality of these articles. Recent studies have examined the published literature base and determined that, in general, the field has progressed beyond an exploratory stage to a point where there is a core set of basic terminology and models. Concurrent with the formation and solidification of a core terminology, there is an increasing emphasis on developing and employing a set of reliable, valid, and reproducible methods for conducting research on operations strategy. We provide a review of common methods for assessing the degree of reliability and agreement of the responses provided by multiple raters within a given organization to a set of qualitative questions. In particular, we examine four methods of determining whether there is evidence of disagreement or bias between multiple raters within a single organization in a mail survey.