Premium
Characterization of the Council of Emergency Medicine Residency Directors' Standardized Letter of Recommendation in 2011–2012
Author(s) -
Love Jeffrey N.,
DeIorio Nicole M.,
RonanBentle Sarah,
Howell John M.,
Doty Christopher I.,
Lane David R.,
Hegarty Cullen
Publication year - 2013
Publication title -
academic emergency medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.221
H-Index - 124
eISSN - 1553-2712
pISSN - 1069-6563
DOI - 10.1111/acem.12214
Subject(s) - medicine , family medicine , pediatric emergency medicine , limiting , emergency department , emergency physician , nursing , mechanical engineering , engineering
Objectives The Council of Emergency Medicine Residency Directors ( CORD ) introduced the standardized letter of recommendation ( SLOR ) in 1997, and it has become a critical tool for assessing candidates for emergency medicine (EM) training. It has not itself been evaluated since the initial studies associated with its introduction. This study characterizes current SLOR use to evaluate whether it serves its intended purpose of being standardized, concise, and discriminating. Methods This retrospective, multi‐institutional study evaluated letters of recommendation from U.S. allopathic applicants to three EM training programs during the 2011–2012 Electronic Residency Application Service ( ERAS ) application cycle. Distributions of responses to each question on the SLOR were calculated, and the free‐text responses were analyzed. Two pilots, performed on five applicants each, assisted in developing a strategy for limiting interrater reliability. Results Each of the three geographically diverse programs provided a complete list of U.S. allopathic applicants to their program. Upon randomization, each program received a list of coded applicants unique to their program randomly selected for data collection. The number of applicants was selected to reach a goal of approximately 200 SLOR s per site ( n = 602). Among this group, comprising 278 of 1,498 applicants (18.6%) from U.S. allopathic schools, a total of 1,037 letters of recommendation were written, with 724 (69.8%) written by emergency physicians. SLOR s represented 57.9% (602/1037) of all LOR s (by any kind of author) and 83.1% (602/724) of letters written by emergency physicians. Three hundred ninety‐two of 602 SLOR s had a single author (65.1%). For the question on “global assessment,” students were scored in the top 10% in 234 of 583 of applications (40.1%; question not answered by some), and 485 of 583 (83.2%) of the applicants were ranked above the level of their peers. Similarly, >95% of all applicants were ranked in the top third compared to peers, for all but one section under “qualifications for emergency medicine.” For 405 of 602 of all SLOR s (67.2%), one or more questions were left unanswered, while 76 of all SLOR s (12.6%) were “customized” or changed from the standard template. Finally, in 291 of 599 of SLOR s (48.6%), the word count was greater than the recommended maximum of 200 words. Conclusions Grade inflation is marked throughout the SLOR , limiting its ability to be discriminating. Furthermore, template customization and skipped questions work against the intention to standardize the SLOR . Finally, it is not uncommon for comments to be longer than guideline recommendations. As an assessment tool, the SLOR could be more discerning, concise, and standardized to serve its intended purpose.