Are Nonprobability Surveys Fit for Purpose?

The article examines the growing use of nonprobability samples (NPSs)—samples where respondents self-select rather than being randomly chosen—in social science survey research, assessing their suitability for various research purposes. While NPSs offer advantages in cost and accessibility, concerns persist about their representativeness, transparency of data collection methods, and potential biases, including fraudulent responses. The authors discuss contexts where NPSs may be appropriate, such as estimating treatment effects in experiments, pretesting survey instruments, studying hard-to-reach populations, and combining with probability samples through Bayesian methods. They emphasize that the proprietary and opaque nature of commercial NPS data limits transparency and challenges the validity of population-level inferences, urging increased investment in probability sampling, enhanced disclosure standards, and greater accountability from survey firms and data users to improve the quality and trustworthiness of survey research.