The Role of Outlier Analysis in Reducing Purposeful Sampling Bias: A Sequential Mixed-Method Approach

Sampling is an important step in researching. Depending on the research question and qualitative or quantitative nature of the study the eligibility and size of the population may vary widely. Due to small size of samples for qualitative analysis, bias may have a larger effect in this type of research where convenient samples are commonly used. The aim of purposefully selected samples is to find information-rich cases allowing in-depth analysis instead of generalizable findings. Using statistical analysis for identifying information-rich cases may reduce bias while allowing qualitative analysis for in-depth research questions. The purpose of this paper is to describe an outlier analysis followed by a cluster analysis to inform purposeful sampling as part of sequential mixed-methods studies. Three hypotheses are tested: 1) Purposeful sampling can be performed using statistical methods that weight criteria equally for all prospective participants. 2) Outliers represent critical cases of groups within a desired population for maximum variation or contrast sampling techniques 3) Due to outlier nature, sample size affects the quality of critical cases identification. The sample included adults in academia and industry who competed a lifelong learning scale and background survey. Using cluster analysis, outliers in four groups were identified based on the interaction between participants’ lifelong learning and STEM background. Two cloud representations were used for increasing confidence in outlier identification, one using raw scales from surveys and other using ranked data from highest to lowest scores. The first method took between-scales variation into account by calculating linkage to the cluster using distance to an elliptical cloud, while the second took that variation into account by ranking values within each scale. The purposeful sample comprised all data points identified as outliers using both strategies. Central tendencies were analyzed to assure that outliers were representing significant differences between groups. This analysis resulted in the identification of outliers with confidence and show statistically that the outliers were part of a sub-cluster, representing a specific group in the population. The study provides a valid and rigorous approach to purposeful sampling, enabling to select a convenient yet unbiased sample. The statistically rigorous selection of participants based in cluster analysis led to a wide variety of cases. This range and representation of sub-groups within a larger population may provide a useful selection of participants for qualitative analyses.