Premium
Novel methods to evaluate fracture risk models
Author(s) -
Donaldson Meghan G,
Cawthon Peggy M,
Schousboe John T,
Ensrud Kristine E,
Lui LiYung,
Cauley Jane A,
Hillier Teresa A,
Taylor Brent C,
Hochberg Marc C,
Bauer Douglas C,
Cummings Steven R
Publication year - 2011
Publication title -
journal of bone and mineral research
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.882
H-Index - 241
eISSN - 1523-4681
pISSN - 0884-0431
DOI - 10.1002/jbmr.371
Subject(s) - frax , osteoporosis , medicine , hip fracture , osteoporotic fracture , femoral neck , fracture (geology) , bone mineral , dentistry , geotechnical engineering , engineering
Fracture prediction models help to identify individuals at high risk who may benefit from treatment. Area under the curve ( AUC ) is used to compare prediction models. However, the AUC has limitations and may miss important differences between models. Novel reclassification methods quantify how accurately models classify patients who benefit from treatment and the proportion of patients above/below treatment thresholds. We applied two reclassification methods, using the National Osteoporosis Foundation (NOF) treatment thresholds, to compare two risk models: femoral neck bone mineral density (BMD) and age (simple model) and FRAX (FRAX model). The Pepe method classifies based on case/noncase status and examines the proportion of each above and below thresholds. The Cook method examines fracture rates above and below thresholds. We applied these to the Study of Osteoporotic Fractures (SOF). There were 6036 (1037 fractures) and 6232 (389 fractures) participants with complete data for major osteoporotic and hip fracture, respectively. Both models for major osteoporotic fracture (0.68 versus 0.69) and hip fracture (0.75 versus 0.76) had similar AUC s. In contrast, using reclassification methods, each model classified a substantial number of women differently. Using the Pepe method, the FRAX model (versus the simple model) missed treating 70 (7%) cases of major osteoporotic fracture but avoided treating 285 (6%) noncases. For hip fracture, the FRAX model missed treating 31 (8%) cases but avoided treating 1026 (18%) noncases. The Cook method (both models, both fracture outcomes) had similar fracture rates above/below the treatment thresholds. Compared with the AUC , new methods provide more detailed information about how models classify patients. © 2011 American Society for Bone and Mineral Research