Premium
Evaluation remains a key point
Author(s) -
MATHYSEN D
Publication year - 2012
Publication title -
acta ophthalmologica
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.534
H-Index - 87
eISSN - 1755-3768
pISSN - 1755-375X
DOI - 10.1111/j.1755-3768.2012.2775.x
Subject(s) - cronbach's alpha , reliability (semiconductor) , multiple choice , value (mathematics) , medicine , test (biology) , significant difference , statistics , psychology , nuclear medicine , mathematics , psychometrics , physics , paleontology , power (physics) , quantum mechanics , biology
Purpose European postgraduate medical assessment have developed during the last 25 years. Currently, all European medical specialties examinations use MCQs. The European Board of Ophthalmology Diploma (EBOD) examination uses multiple independent true/false MCQs for the written part of the examination. Since true/false MCQs may be prone to guessing, thorough statistical evaluation has been set‐up over the last five years to monitor the performance of the examination and its MCQs. Methods In 2010, the European Board of Ophthalmology (EBO) decided to introduce negative marks for the MCQs. To study the influence of negative marks on the performance/reliability of the examination, the following statistical performance parameters of test items have been compared: P‐value, Rit‐value, Cronbach‐alpha and 3‐parameter item‐response analysis. Results A decrease in average P‐value (P < 0.66) was observed compared to the situation without negative marks (P > 0.75). An increase in average Rit‐value (Rit > 0.15) was observed compared to the situation without negative marks (Rit < 0.15). An increase in Cronbach‐alpha (> 0.85) was observed compared to the situation without negative marks (< 0.80). 3‐parameter item‐response analyses revealed that almost none of the questions was influenced by guessing correctly (average c <<< 0.33), while without negative marks all questions were influenced by guessing correctly (average c ~ 0.33). Conclusion Introduction of negative marks at the EBO examination, did lead to a decrease in item‐facility (P‐value) due to less "wild" guessing; an increase in Pearson correlation (Rit‐value) between item and total scores and an increase in reliability (Cronbach‐alpha). 3‐parameter analysis showed that the portion of correct answers due to guessing is negligible with the use of negative marks.