Detecting “real” population changes with American Community Survey data: The implicit assumption of treating between-year differences as “trends”

BACKGROUND:  The American Community Survey (ACS) in the United States (US) collects detailed demographic information on the US population. Pressures to use year-to-year population estimates to analyze “trends” (i.e., between-year differences on the characteristics of interest) have motivated the need to explore how single- or multi-year estimates can be used to investigate changes in US population over time. OBJECTIVE: The specific aim of this manuscript is to provide empirical evidence that between-year differences in population characteristics have difference levels of uncertainty around point-estimates. METHODS: Six ACS Public Use Microdata Sample (PUMS) single year files from 2005 through 2010 are used to empirically show the heterogeneity of uncertainty in “between-year differences” on level of education, for a birth cohort born between 1960 and 1970 of non-Latino-whites and Mexican Latinos/as. RESULTS: The data show the precision of the education estimate decreases as the specificity of the population increases. For example, Mexican’s 99% confidence intervals have wider and more time-varying bandwidths than non-Latino-whites. CONCLUSIONS: Inferring meaningful population change requires the challengeable assumption that between-year differences are not the product of data artifacts. Harvesting reputable ACS data demands further research before between-year differences can be treated as “real change.”