How to Create a Bad Survey Instrument

A s educators and education researchers, we often survey trainees, faculty, and patients as a rapid and accurate method to obtain data on outcomes of interest. The Accreditation Council for Graduate Medical Education surveys residents and faculty every year; institutions survey graduating residents and staff regarding learning environments; program directors survey residents about rotation experiences and faculty skills; and researchers use surveys to measure a range of outcomes, from empathy to well-being to patient satisfaction. As editors, we see survey instruments in submitted manuscripts daily. These include questionnaires previously used in other studies, and others that are ‘‘homegrown.’’ In a 2012 review of papers submitted to this journal, 77% used a survey instrument to assess at least 1 outcome, and a more recent study of 3 high-impact medical education journals found that 52% of research studies used at least 1 survey. Despite advice from many sources, including this journal, we continue to see manuscript submissions with surveys unlikely to yield reliable or valid data. We suggest that if you want to create a dubious, lowquality survey, follow the tips in bold below (and see the TABLE). If not, read further about each item. Remember that creating a credible survey may take more time and effort, but it is well worth the investment.