Power Analysis Tutorial for Experimental Design Software

: Statistical power calculations for designed experiments are essential to right-sizing tests during planning. Under-sized tests will fail to uncover true contributors affecting system effectiveness and suitability, while over-sized tests are wasteful. Although the concepts of statistical power are reasonably well understood, the mechanics of computations are not necessarily well publicized. The statistical software packages are not necessarily consistent in requesting user information, nor are they clear or consistent in the assumptions made for the necessary power information not requested. Most likely the least understood concept and the one software companies fail to agree upon is the method for sizing effects for categorical factors with more than two levels. These effects are critical ingredients to the power equation. This document reviews basic statistical power concepts as they relate to the design of experiments, discuss the differences between and the proper steps for continuous response variable power versus binary response power, describe the power formulation intricacies for designs involving multi-level categorical factors, and finally to compare software platform interfaces and power computation differences. The intent is to make you aware of the differences in power estimates across software packages, but even more importantly to equip you to confidently and successfully estimate power for your testing.