Critical Evaluation of Models and Their Parameters

Some form of critical evaluatory procedure for models is needed to maintain the integrity of modeling and to ensure that the increasingly widespread use of models does not result in the propagation of misleading information. The term validation must be used with the clear understanding that no model can be validated in the sense that it has been unequivocally justified. All that can be achieved is to show how small the probability is that the model has been refuted. Whether this probability is acceptable is a subjective decision. The type of statistical test that is appropriate depends on the quality of the data against which the model is tested. Using a procedure that compares the sums of squares that result from the model not fitting the data with the sums of squares due to error in the data, gives a stringent test, but it requires the replication of measurements. Rigor is as important in evaluating parameters as it is in testing models. Direct measurement is the best option, but where a parameter has to be obtained by fitting, the statistical procedures used for validation are appropriate. In general, the further the data used for parameterization are removed from the data to be simulated the better. Problems can arise in both parameterization and validation if the model is nonlinear regarding its parameters, and the latter have appreciable variances. Parameterization and validation become more difficult as the complexity of the model or the scale at which it is used increase.