Premium
Generalizability analysis for clinical trials: a simulation study
Author(s) -
Wang Wei,
Ma Ying,
Huang Yangxin,
Chen Henian
Publication year - 2017
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.7238
Subject(s) - generalizability theory , clinical trial , medicine , population , selection bias , random effects model , generalization , statistics , covariate , treatment effect , demography , mathematics , meta analysis , mathematical analysis , environmental health , sociology , traditional medicine
Subjects are rarely selected on a random basis from a well‐defined patient population of interest into a clinical trial, with women, children, the elderly, and those with common comorbidities who are frequently underrepresented. Decades of clinical experience have demonstrated that the application of trial findings to individual patients is permissible by using efficacy as a measure of effectiveness and assuming that the characteristics of patients are sufficiently similar. In order to investigate this issue in greater depth, we simulated a patient population with treatment effect size of 0.5 (Cohen's d ) and five covariates that included gender, health insurance, comorbidity, age, and motivation. To demonstrate how selection of patients for a clinical trial can bias the results when treatment effect varies across individuals, we created 50 nonrandom clinical trials based on this patient population and showed relative bias to range from 1.68% to 99.70%. We calculated and evaluated three indexes: C‐statistics, standardized mean difference (SMD), and Tipton's index ( β ) of generalization for the 50 nonrandom trials. Findings indicated that (i) the ranges were 0.56–0.98, 0.23–11.17, and 0.99–0.73 for C‐statistics, SMD, and β , respectively, when treatment effect bias increased from 1.68% to 99.70% and (ii) C‐statistics < 0.86, SMD < 1.95, and β > 0.91 when treatment effect bias <50%. Recommendations are made using existing generalization indexes on the basis of our simulation results. An example from a real clinical trial is provided for illustration. Copyright © 2017 John Wiley & Sons, Ltd.