Testing strategies for simulation optimization

There is increasing interest in science and industry in the optimization of computer simulation models. Often these models are not Monte-Carlo simulations, but consist of systems of differential equations, or other mathematical models. These models can present special problems to numerical optimization methods. First, derivatives are often unavailable. Second, function evaluations can be extremely expensive (e.g. 1 hour on an IBM 3090). Third, the numerical accuracy of each function value may depend on a complicated chain of calculations, and so be impractical to pre-specify. This last point makes it difficult to calibrate optimization routines that use finite difference approximations for gradients. This paper presents a strategy for comparing optimization techniques for these problems, and reviews several interesting findings for quasi-Newton methods, simplex search, and others.