Selecting the best treatment in designed experiments

A common question in biological experimentation is concerned with finding an optimal level for a treatment using data obtained from a designed experiment. For example, an experiment might concern a group of hyperactive adolescent males that have been grouped into blocks according to some relevant factors. Within each group the subjects are randomly assigned to several treatments for hyperactivity. An important aspect of the study would then be determining which of the treatment options is most effective in reducing hyperactivity in the subjects. Basing such a determination directly on the estimated treatment effects may be unreliable because of the inherent variability associated with the estimates. Other methods, such as multiple comparison procedures and best subset selection procedures can also be applied to this problem. Unfortunately, such procedures are often difficult to interpret and rely on assumptions that may not be appropriate for the experiment under consideration. In this paper we develop a method of assigning a confidence level to each treatment that measures how confident we are that each treatment is optimal. The method for assigning the confidence levels is based on a more general methodology developed for the problem of regions, where one assigns levels of confidence that an unknown parameter is within specified subsets of the parameter space. The actual computation of the confidence levels is based on the bootstrap. The method is applied to an example and is studied empirically. Copyright © 2003 John Wiley & Sons, Ltd.