Premium
An Empirical Validation Study of Popular Survey Methodologies for Sensitive Questions
Author(s) -
Rosenfeld Bryn,
Imai Kosuke,
Shapiro Jacob N.
Publication year - 2016
Publication title -
american journal of political science
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 6.347
H-Index - 170
eISSN - 1540-5907
pISSN - 0092-5853
DOI - 10.1111/ajps.12205
Subject(s) - referendum , social desirability bias , survey data collection , non response bias , response bias , social psychology , psychology , political science , survey methodology , econometrics , public economics , statistics , economics , social desirability , law , politics , mathematics
When studying sensitive issues, including corruption, prejudice, and sexual behavior, researchers have increasingly relied upon indirect questioning techniques to mitigate such known problems of direct survey questions as underreporting and nonresponse. However, there have been surprisingly few empirical validation studies of these indirect techniques because the information required to verify the resulting estimates is often difficult to access. This article reports findings from the first comprehensive validation study of indirect methods. We estimate whether people voted for an anti‐abortion referendum held during the 2011 Mississippi General Election using direct questioning and three popular indirect methods: list experiment, endorsement experiment, and randomized response. We then validate these estimates against the official election outcome. While direct questioning leads to significant underestimation of sensitive votes against the referendum, indirect survey techniques yield estimates much closer to the actual vote count, with endorsement experiment and randomized response yielding the least bias.