Premium
Sample size considerations and predictive performance of multinomial logistic prediction models
Author(s) -
Jong Valentijn M. T.,
Eijkemans Marinus J. C.,
Calster Ben,
Timmerman Dirk,
Moons Karel G. M.,
Steyerberg Ewout W.,
Smeden Maarten
Publication year - 2019
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.8063
Subject(s) - multinomial logistic regression , statistics , multinomial distribution , sample size determination , predictive modelling , context (archaeology) , logistic regression , econometrics , overfitting , computer science , mathematics , artificial intelligence , paleontology , artificial neural network , biology
Multinomial Logistic Regression (MLR) has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes. We present a full‐factorial simulation study to examine the predictive performance of MLR models in relation to the relative size of outcome categories, number of predictors and the number of events per variable. It is shown that MLR estimated by Maximum Likelihood yields overfitted prediction models in small to medium sized data. In most cases, the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized MLR. Our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size. As expected, our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model. We recommend the use of penalized MLR when prediction models are developed in small data sets or in medium sized data sets with a small total sample size (ie, when the sizes of the outcome categories are balanced). Finally, we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer.