Premium
Performance Measures for Selection of Metamodels to be Used in Simulation Optimization
Author(s) -
Keys Anthony C.,
Rees Loren Paul,
Greenwood Allen G.
Publication year - 2002
Publication title -
decision sciences
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.238
H-Index - 108
eISSN - 1540-5915
pISSN - 0011-7315
DOI - 10.1111/j.1540-5915.2002.tb01635.x
Subject(s) - computer science , selection (genetic algorithm) , context (archaeology) , mathematical optimization , machine learning , mathematics , paleontology , biology
This paper points out the need for performance measures in the context of simulation optimization and suggests six such measures. Two of the measures are indications of absolute performance, whereas the other four are useful in assessing the relative performance of various candidate metamodels. The measures assess performance on three fronts: accuracy of placing optima in the correct location, fit to the response, and fit to the character of the surface (expressed in terms of the number of optima). Examples are given providing evidence of the measures' utility—one in a limited scenario deciding which of two competing metamodels to use as simulation optimization response surfaces vary, and the other in a scenario of a researcher developing a new, sequential optimization search procedure.