The Influence of Response Mode on Study Results: Offering Cigarette Smokers a Choice of Postal or Online Completion of a Survey

Background It is unclear whether offering online data collection to study participants affects compliance or produces bias. Objective To compare response rates, baseline characteristics, test-retest reliability, and outcomes between cigarette smokers who chose to complete a survey by mail versus those who chose to complete it online. Methods We surveyed cigarette smokers who intended to stop smoking within the next 30 days to determine barriers to calling a smoking quit line. Participants were offered the choice of completing a paper version of the survey sent through the mail or an online version at a password-protected website. Participants were called 2 months later to determine if they had made a quit attempt and/or called a smoking quit line since the baseline survey. We compared characteristics and outcomes among those who chose postal versus online completion. We measured test-retest reliability of the baseline survey by resurveying a semirandom sample of participants within 10 days of the original survey. Results Of 697 eligible respondents to newspaper ads in 12 US cities, 438 (63%) chose to receive a mailed paper survey and 259 (37%) chose an Internet survey. Survey return rates were the same for the 2 modes (92% versus 92%, P = .82). Online respondents were younger (mean of 46 versus 51 years old for postal, P < .001), more likely to be white (76% versus 62%, P < .001), less likely to be African American (18% versus 30%, P < .001), more highly educated (34% college graduate versus 23%, P < .001), more likely to intend to stop smoking in the next 30 days (47% definitely versus 30%, P < .001), and more likely to have heard of a smoking quit line (51% versus 40%, P = .008). Participants did not differ on gender (54% female for online versus 55% for postal, P = .72) or cigarettes smoked per day (mean of 19 versus 21, P = .30). Online respondents had slightly fewer missing items on the 79-item survey (mean of 1.7% missing versus 2.3%, P = .02). Loss to follow-up at 2 months was similar (16% for online and 15% for postal, P = .74). There was no significant difference between online and postal respondents in having called a smoking quit line during the 2-month follow-up period (20% versus 24%, P = .22) or in having made a quit attempt (76% versus 79%, P = .41). Conclusions Cigarette smokers who chose to complete a survey using the Internet differed in several ways from those who chose mailed surveys. However, more importantly, online and postal responses produced similar outcomes.