Premium
Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility
Author(s) -
Paul Ram,
Richard Grol,
JanJoost Rethans,
Bojoura Schouten,
Cees van der Vleuten,
A.D.M. Kester
Publication year - 1999
Publication title -
medical education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.776
H-Index - 138
eISSN - 1365-2923
pISSN - 0308-0110
DOI - 10.1046/j.1365-2923.1999.00348.x
Subject(s) - generalizability theory , reliability (semiconductor) , logbook , sample (material) , global positioning system , validity , psychology , general practice , medicine , applied psychology , quality (philosophy) , medical education , family medicine , psychometrics , clinical psychology , computer science , developmental psychology , power (physics) , oceanography , physics , chemistry , quantum mechanics , epistemology , geology , telecommunications , philosophy , chromatography
Objectives To develop a video assessment method for General Practitioners (GPs) by analysing issues of validity, reliability and feasibility of observation of videotaped regular consultations. Design In a cross‐sectional study consultations of 93 GPs were video recorded in the practice during 1 week. The GPs registered consultation and patient data in a logbook; 16 consultations per GP were selected using preset criteria. The quality of communicative and medical performance of these consultations was assessed by GP observers with a validated instrument. The validity of the procedure was evaluated by checking the content of each GP’s sample using specific sample criteria. Selection bias was estimated by multiple regression analysis, with sample characteristics as independent variables and scores on communication and medical performance as dependent variables. The influence of observation on GPs and patients was assessed by a questionnaire. Generalizability theory was used to estimate reliability. Feasibility was assessed by conducting a questionnaire, by keeping accounts, and by checking the technical quality of the videotaped consultations. Setting Universities of Nijmegen and Maastricht, The Netherlands. Subjects General Practitioners (GPs). Results The domain of general practice was well covered in the samples; content validity was satisfactory. With regard to the sample characteristics, only the total duration of consultations appeared to correlate significantly with both the score on communication and the score on medical performance. A majority (71%) of GPs reported not being influenced by the observation, except in the first cases, and recognizing their usual daily performance in the videotaped consultations. An acceptable level of reliability was reached after 2·5 hours of observation, i.e. 12 cases by a single observer. The method was well accepted by both GPs and patients. The costs were £250 per GP. Conclusions Video assessment of GPs in daily practice according to the procedures described is a valid and reliable method, one which is useful for education and quality improvement. There is a trade‐off between feasibility on one hand and validity, reliability and credibility on the other hand. Compared to investments in observation methods in standardized settings, the costs of video observation of GPs’ actual performance are acceptable.