Premium
Development of a novel assessment tool for higher order learning in physiology: a utility analysis (719.11)
Author(s) -
Kibble Jonathan,
Cramer Nicholas
Publication year - 2014
Publication title -
the faseb journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.709
H-Index - 277
eISSN - 1530-6860
pISSN - 0892-6638
DOI - 10.1096/fasebj.28.1_supplement.719.11
Subject(s) - rubric , workload , context (archaeology) , medical education , test (biology) , psychology , construct validity , medical diagnosis , higher order thinking , construct (python library) , mathematics education , computer science , teaching method , medicine , clinical psychology , psychometrics , pathology , paleontology , cognitively guided instruction , biology , operating system , programming language
Multiple‐choice questions (MCQs) are a gold‐standard tool for assessment of knowledge and are used in many high‐stakes standardized tests. However, MCQs are frequently criticized for lacking the ability to test higher order learning. Our objective was to develop a novel assessment to assess learning at the levels of application, evaluation and synthesis. The rubric assesses student write‐ups of clinical case problems. The method is based on the traditional doctor’s note, called “SOAP” (in our case: “Subjective and Objective Assessment of Pathophysiology/Pharmacology). Students are required to correctly identify subjective and objective findings, to ascribe pathophysiologic and pharmacologic mechanisms, and to justify a list of differential diagnoses. The study received IRB approval. Analysis was applied to de‐identified records of 80 second year medical students from three clinical case studies. Each script was double‐marked by a pair of faculty evaluators. A utility analysis was undertaken by combining numeric data with the results of faculty and student interviews. The rubric was found to display high levels of content validity and construct validity and has acceptable levels of statistical reliability. Current limitations include high time demands on expert graders and workload concerns from students. In conclusion, the novel SOAPP instrument offers potential as a method for assessing higher order learning in an authentic learning context.