Premium
Quantifying how small variations in design elements affect risk in an incident cohort study in claims
Author(s) -
Izem Rima,
Huang TingYing,
Hou Laura,
Pestine Ella,
Nguyen Michael,
Maro Judith C.
Publication year - 2020
Publication title -
pharmacoepidemiology and drug safety
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.023
H-Index - 96
eISSN - 1099-1557
pISSN - 1053-8569
DOI - 10.1002/pds.4892
Subject(s) - medicine , statistics , cohort study , matching (statistics) , propensity score matching , clinical study design , replicate , estimation , cohort , hazard ratio , risk assessment , econometrics , risk analysis (engineering) , computer science , mathematics , confidence interval , clinical trial , engineering , computer security , systems engineering
Background Epidemiological study reporting is improving but is not transparent enough for easy evaluation or replication. One barrier is insufficient details about design elements in published studies. Methods Using a previously conducted drug safety evaluation in claims as a test case, we investigated the impact of small changes in five key design elements on risk estimation. These elements are index day of incident exposure's determination of look‐back or follow‐up periods, exposure duration algorithms, heparin exposure exclusion, propensity score model variables, and Cox proportional hazard model stratification. We covaried these elements using a fractional factorial design, resulting in 24 risk estimates for one outcome. We repeated eight of these combinations for two additional outcomes. We measured design effects on cohort sizes, follow‐up time, and risk estimates. Results Small changes in specifications of index day and exposure algorithm affected the risk estimation process the most. They affected cohort size on average by 8 to 10%, follow‐up time by up to 31%, and magnitude of log hazard ratios by up to 0.22. Other elements affected cohort before matching or risk estimate's precision but not its magnitude. Any change in design substantially altered the matched control‐group subjects in 1:1 matching. Conclusions Exposure‐related design elements require attention from investigators initiating, evaluating, or wishing to replicate a study or from analysts standardizing definitions. The methods we developed, using factorial design and mapping design effect on causal estimation process, are applicable to planning of sensitivity analyses in similar studies.