English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Many published scale validation studies determine inter‐rater reliability using the intra‐class correlation coefficient  ( ICC ) . However, the use of this statistic must consider its advantages, limitations, and applicability. This paper evaluates how interaction of subject distribution, sample size, and levels of rater disagreement affects   ICC   and provides an approach for obtaining relevant   ICC   estimates under suboptimal conditions. Simulation results suggest that for a fixed number of subjects,   ICC   from the convex distribution is smaller than   ICC   for the uniform distribution, which in turn is smaller than   ICC   for the concave distribution. The variance component estimates also show that the dissimilarity of   ICC   among distributions is attributed to the study design (ie, distribution of subjects) component of subject variability and not the scale quality component of rater error variability. The dependency of   ICC   on the distribution of subjects makes it difficult to compare results across reliability studies. Hence, it is proposed that reliability studies should be designed using a uniform distribution of subjects because of the standardization it provides for representing objective disagreement. In the absence of uniform distribution, a sampling method is proposed to reduce the non‐uniformity. In addition, as expected, high levels of disagreement result in low   ICC  , and when the type of distribution is fixed, any increase in the number of subjects beyond a moderately large specification such as   n  = 80  does not have a major impact on   ICC  .

Performance of intraclass correlation coefficient (ICC) as a reliability index under various distributions in scale reliability studies