Premium
Intraclass Correlation Coefficient ( ICC ): A Framework for Monitoring and Assessing Performance of Trained Sensory Panels and Panelists
Author(s) -
Bi Jian,
Kuesten Carla
Publication year - 2012
Publication title -
journal of sensory studies
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.61
H-Index - 53
eISSN - 1745-459X
pISSN - 0887-8250
DOI - 10.1111/j.1745-459x.2012.00399.x
Subject(s) - intraclass correlation , reliability (semiconductor) , ranking (information retrieval) , sensory system , similarity (geometry) , statistics , computer science , estimator , data mining , sensory analysis , mathematics , artificial intelligence , psychology , psychometrics , power (physics) , physics , quantum mechanics , image (mathematics) , cognitive psychology
Sensory measurement underlies sensory science. Sensory analysis and decision‐making heavily depend on the quality of sensory data, which is determined by the performance of trained sensory panels and panelists. Various methods have been proposed for monitoring and assessing the performance. A weakness of the currently used methods is lack of a unified framework for various criteria and a variety of experiments with different types of data. This paper proposes to use accuracy, validity and reliability as general terminologies to describe sensory measurement and to apply the intraclass correlation coefficient ( ICC ) as a framework for monitoring and assessing performance. ICC can measure both similarity among panelists and sensitivity of panels and panelists. Hence, ICC can handle the problems of both reliability and validity. ICC can be obtained from different types of data for diverse experiments. This paper provides the equations and R and S ‐ P lus functions for estimations of ICC s from continuous data (ratings), multivariate continuous data, ordinal data, ranking data, binary‐choice data, multiple‐choice data and forced‐choice data. Confidence intervals, variances of the estimators, comparison with a fixed value and difference and similarity tests for multiple ICC s are also provided. The relationship between C ronbach's coefficient alpha and ICC is discussed. Practical Applications Intraclass correlation coefficient ( ICC ) may play a framework role in monitoring and assessing performance of trained sensory panels and panelists. It can be used as an index of the quality of sensory data. The larger the ICC or C ronbach's coefficient alpha value, the better the performance of panels and panelists. If and only if an ICC is significantly larger than a specified value, can the quality of the data for that attribute be regarded as acceptable. A statistical test that fails to show that an ICC or C ronbach's coefficient alpha is significantly larger than a specified lowest limit, e.g., 0.1, suggests that the products are undiscriminating, or the sensory data, at least for that attribute, might not be valid and reliable. We should use that data with caution.