The Problem with Online Data Collection: Predicting Invalid Responding in Undergraduate Samples

The popularity of online research is increasing, but the validity of the results obtained is not yet clear. The purpose of this study was to examine some of the factors that influence the validity of computerized data collection in an undergraduate sample. Participants were 99 university students randomly assigned to one of three data collection conditions: online survey platform, in-person computerized survey platform, and in-person pencil-and-paper survey. Results suggest self-reported inattention symptoms, exposure to more stressors, and computerized platforms predicted more invalid responding. In contrast, personality, self-reported impulsivity symptoms, and shorter completion times did not predict invalid responding. Overall, more than half of the participants failed at least one validity check and 11% failed three or more validity checks. Researchers, particularly those working with undergraduate samples, should consider implementing procedures to ensure the data collected online are valid.