Tactical defenses against systematic variation in wind tunnel testing

This paper examines the role of unexplained systematic variation on the reproducibility of wind tunnel test results. Sample means and variances estimated in the presence of systematic variations are shown to be susceptible to bias errors that are generally non-reproducible functions of those variations. Unless certain precautions are taken to defend against the effects of systematic variation, it is shown that experimental results can be difficult to duplicate and of dubious value for predicting system response with the highest precision or accuracy that could otherwise be achieved. Results are reported from an experiment designed to estimate how frequently systematic variations are in play in a representative wind tunnel experiment. These results suggest that significant systematic variation occurs frequently enough to cast doubts on the common assumption that sample observations can be reliably assumed to be independent. The consequences of ignoring correlation among observations induced by systematic variation are considered in some detail. Experimental tactics are described that defend against systematic variation. The effectiveness of these tactics is illustrated through computational experiments and real wind tunnel experimental results. Some tutorial information describes how to analyze experimental results that have been obtained using such quality assurance tactics.