Premium
Inter‐rater reliability of “clean cup” scores by coffee experts
Author(s) -
Giacalone Davide,
Steen Ida,
Alstrup Jesper,
Münchow Morten
Publication year - 2020
Publication title -
journal of sensory studies
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.61
H-Index - 53
eISSN - 1745-459X
pISSN - 0887-8250
DOI - 10.1111/joss.12596
Subject(s) - reliability (semiconductor) , quality (philosophy) , context (archaeology) , point (geometry) , control (management) , variety (cybernetics) , computer science , psychology , applied psychology , artificial intelligence , mathematics , paleontology , philosophy , power (physics) , physics , geometry , epistemology , quantum mechanics , biology
Cupping scores from experts are extensively used in the coffee industry for a variety of applications, from quality control to judging coffee competitions. In this paper, we examined inter‐rater reliability (IRR) of “clean cup” ratings by coffee experts (“cuppers”) in two studies. In both studies, IRR reliability was found to be low, denoting a lack of concept alignment among experts. Remarkably, however, within‐assessor reproducibility was high, suggesting that expert cuppers have their own individual understanding of “clean cup.” Practical applications The results presented suggested that “clean cup” scores have a fundamentally subjective nature. Since cupping scores are routinely used to drive business decisions (particularly in the context of quality control), it would be advisable that such attributes be anchored in a precise definition (in the case of “clean cup” of what constitute a defect from a sensory point of view) developed based on properly conducted sensory studies.