Premium
AUC: a misleading measure of the performance of predictive distribution models
Author(s) -
Lobo Jorge M.,
JiménezValverde Alberto,
Real Raimundo
Publication year - 2008
Publication title -
global ecology and biogeography
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.164
H-Index - 152
eISSN - 1466-8238
pISSN - 1466-822X
DOI - 10.1111/j.1466-8238.2007.00358.x
Subject(s) - receiver operating characteristic , measure (data warehouse) , statistics , reliability (semiconductor) , goodness of fit , model selection , environmental niche modelling , mathematics , variable (mathematics) , econometrics , computer science , data mining , ecology , ecological niche , biology , mathematical analysis , power (physics) , physics , quantum mechanics , habitat
The area under the receiver operating characteristic (ROC) curve, known as the AUC, is currently considered to be the standard method to assess the accuracy of predictive distribution models. It avoids the supposed subjectivity in the threshold selection process, when continuous probability derived scores are converted to a binary presence–absence variable, by summarizing overall model performance over all possible thresholds. In this manuscript we review some of the features of this measure and bring into question its reliability as a comparative measure of accuracy between model results. We do not recommend using AUC for five reasons: (1) it ignores the predicted probability values and the goodness‐of‐fit of the model; (2) it summarises the test performance over regions of the ROC space in which one would rarely operate; (3) it weights omission and commission errors equally; (4) it does not give information about the spatial distribution of model errors; and, most importantly, (5) the total extent to which models are carried out highly influences the rate of well‐predicted absences and the AUC scores.