Error assessment in decision-tree models applied to vegetation analysis

Methods were developed to evaluate the performance of a decision-tree model used to predict landscape-level patterns of potential forest vegetation in central New York State. The model integrated environmental databases and knowledge on distribution of vegetation. Soil and terrain decision-tree variables were derived by processing state-wide soil geographic databases and digital terrain data. Variables used as model inputs were soil parent material, soil drainage, soil acidity, slope position, slope gradient, and slope azimuth. Landscapescale maps of potential vegetation were derived through sequential map overlay operations using a geographic information system (GIS). A verification sample of 276 field plots was analyzed to determine: (1) agreement between GIS-derived estimates of decision-tree variables and direct field measurements, (2) agreement between vegetation distributions predicted using GIS-derived estimates and using field observations, (3) effect of misclassification costs on prediction agreement, (4) influence of particular environmental variables on model predictions, and (5) misclassification rates of the decision-tree model. Results indicate that the prediction model was most sensitive to drainage and slope gradient, and that the imprecision of the input data led to a high frequency of incorrect predictions of vegetation. However, in many cases of misclassification the predicted vegetation was similar to that of the field plots so that the cost of errors was less than expected from the misclassification rate alone. Moreover, since common vegetation types were more accurately predicted than rare types, the model appears to be reasonably good at predicting vegetation for a randomly selected plot in the landscape. The error assessment methodology developed for this study provides a useful approach for determining the accuracy and sensitivity of landscape-scale environmental models, and indicates the need to develop appropriate field sampling procedures for verifying the predictions of such models.