Premium
Incorporation of Core Competency Questions Into an Annual National Self‐Assessment Examination for Residents in Physical Medicine and Rehabilitation: Results and Implications
Author(s) -
Webster Joseph B.
Publication year - 2009
Publication title -
pmandr
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.617
H-Index - 66
eISSN - 1934-1563
pISSN - 1934-1482
DOI - 10.1016/j.pmrj.2008.11.008
Subject(s) - test (biology) , medicine , core competency , reliability (semiconductor) , rehabilitation , core (optical fiber) , medical education , physical therapy , family medicine , psychology , gerontology , computer science , telecommunications , quantum mechanics , paleontology , power (physics) , physics , biology , marketing , business
Objective To determine the performance and change over time when incorporating questions in the core competency domains of practice‐based learning and improvement (PBLI), systems‐based practice (SBP), and professionalism (PROF) into the national PM&R Self‐Assessment Examination for Residents (SAER). Design Prospective, longitudinal analysis. Setting The national Self‐Assessment Examination for Residents (SAER) in Physical Medicine and Rehabilitation, which is administered annually. Participants Approximately 1100 PM&R residents who take the examination annually. Interventions Inclusion of progressively more challenging questions in the core competency domains of PBLI, SBP, and PROF. Main Outcome Measurements Individual test item level of difficulty ( P value) and discrimination (point biserial index). Results Compared with the overall test, questions in the subtopic areas of PBLI, SBP, and PROF were relatively easier and less discriminating (correlation of resident performance on these domains compared with that on the total test). These differences became smaller during the 3‐year time period. The difficulty level of the questions in each of the subtopic domains was raised during the 3 year period to a level close to the overall exam. Discrimination of the test items improved or remained stable. Conclusions This study demonstrates that, with careful item writing and review, multiple‐choice items in the PBLI, SBP, and PROF domains can be successfully incorporated into an annual, national self‐assessment examination for residents. The addition of these questions had value in assessing competency while not compromising the overall validity and reliability of the exam. It is yet to be determined if resident performance on these questions corresponds to performance on other measures of competency in the areas of PBLI, SBP, and PROF.