Dental Licensure Exams: Human Subjects and One‐Shot Evaluations

nation of competency to begin independent dental practice can only be achieved based on testing involving patients because that is what dentists do. In fact such testing should be in realistic contexts, including diagnosis and management of long-term, comprehensive care. The Chambers and Loos study was written to demonstrate that one-shot evaluation is too unreliable to support high-stakes professional decisions. The test format is correct. It just has to be used several times instead of once. The impression that rater calibration matters has been perpetuated by failure to measure the trial-to-trial variation in testing. Context specific , patient-related variance overwhelms other considerations. Chambers and Loos report methods for correcting this shortcoming. A valid evaluation system with low reliability can be augmented to achieve acceptable reliability through repeated testing. A testing scheme with weak validity cannot be made more valid, either through repeated testing or enhanced reliability. The flaw in the current approach to initial li-censure examination is not the patient; it is the one-shot nature of the data collection. A defensible decision about licensing candidates requires multiple measures of their performance performing comprehensive patient care. The Association Report in the October journal is a reasoned and constructive advance over the politically defensive and psychometrically opaque debates of the past half-century of calls for initial licen-sure report. Any steps on that path would be progress. I would only ask for careful consideration and clarification of a single phrase in the first of the report's recommendations: " The criteria agreed upon unanimously by all members of the committee [AADE-ADEA Innovative Testing and Educational Methodologies Committee] are that the clinical li-censure process should: 1) Be a process administered by independent third parties occurring within the educational process. " The phrase " administered by " ad-Dear Dr. Alvares: I found myself and one of my colleagues quoted extensively in " A Survey of Deans and ADEA Activities on Dental Licensure Issues, " an Association Report by Drs. Ranney, Haden, Weaver, and Valachovic in the October 2003 issue of the Journal of Dental Education. 1 The facts of our research (in Chambers and Loos, " Analyzing the Sources of Unreliability in Fixed Prosthodontics Mock Board Examinations " 2) were correctly reported. A single clinical test involving patients is inherently unreliable (r-values in the .10 to .20 range) and cannot be corrected by any feasible combination of improved examiner calibration or increases in the number of …