Premium
Two Criteria for Evaluating Risk Prediction Models
Author(s) -
Pfeiffer R. M.,
Gail M. H.
Publication year - 2011
Publication title -
biometrics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 2.298
H-Index - 130
eISSN - 1541-0420
pISSN - 0006-341X
DOI - 10.1111/j.1541-0420.2010.01523.x
Subject(s) - population , statistics , inference , incidence (geometry) , disease , medicine , mathematics , risk assessment , computer science , demography , econometrics , environmental health , artificial intelligence , geometry , computer security , sociology
Summary We propose and study two criteria to assess the usefulness of models that predict risk of disease incidence for screening and prevention, or the usefulness of prognostic models for management following disease diagnosis. The first criterion, the proportion of cases followed PCF ( q ) , is the proportion of individuals who will develop disease who are included in the proportion q of individuals in the population at highest risk. The second criterion is the proportion needed to follow‐up, PNF ( p ) , namely the proportion of the general population at highest risk that one needs to follow in order that a proportion p of those destined to become cases will be followed. PCF ( q ) assesses the effectiveness of a program that follows 100 q % of the population at highest risk. PNF ( p ) assess the feasibility of covering 100 p % of cases by indicating how much of the population at highest risk must be followed. We show the relationship of those two criteria to the Lorenz curve and its inverse, and present distribution theory for estimates of PCF and PNF . We develop new methods, based on influence functions, for inference for a single risk model, and also for comparing the PCF s and PNF s of two risk models, both of which were evaluated in the same validation data.