English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

In workplace-based assessment, research has suggested that aligning rating scales with how clinical supervisors naturally conceptualize trainee performance improves reliability and makes assessment more efficient. This study examined the generalizability of those findings by determining if construct alignment improves the reliability of competency-focused program evaluation ratings provided by 135 Year 3 medical students and 185 faculty supervisors. Participants were randomly assigned to complete the Readiness for Clerkship survey using one of two construct-aligned rating scales or a competence-focused rating scale. Mean differences, Spearman correlations, and Generalizability analyses compared the scales’ utility. Relative to the competence-oriented scale, the construct-aligned scales showed greater consistency in the rank order of items between students and faculty. Eight student raters were needed to achieve G=0.8 using the independence scale, 12 were needed using the behavioral/independence scale, and 22 were needed using the competence-oriented scale. Faculty were less consistent in their ratings and required more raters (k=25-32), regardless of the scale used.  These results extend previous research into the benefits of construct-aligned scales by suggesting that students offer better ratings when the scales used to collect their judgments are aligned with the constructs of independence and sophistication rather than being phrased in terms of competence.

Construct-Aligned Rating Scales Improve the Reliability of Program Evaluation Data