Results of Multivariable Logistic Regression, Propensity Matching, Propensity Adjustment, and Propensity-based Weighting under Conditions of Nonuniform Effect

Observational studies often provide the only available information about treatment effects. Control of confounding, however, remains challenging. The authors compared five methods for evaluating the effect of tissue plasminogen activator on death among 6,269 ischemic stroke patients registered in a German stroke registry: multivariable logistic regression, propensity score-matched analysis, regression adjustment with the propensity score, and two propensity score-based weighted methods-one estimating the treatment effect in the entire study population (inverse-probability-of-treatment weights), another in the treated population (standardized-mortality-ratio weights). Between 2000 and 2001, 212 patients received tissue plasminogen activator. The crude odds ratio between tissue plasminogen activator and death was 3.35 (95% confidence interval: 2.28, 4.91). The adjusted odds ratio depended strongly on the adjustment method, ranging from 1.11 (95% confidence interval: 0.67, 1.84) for the standardized-mortality-ratio weighted to 10.77 (95% confidence interval: 2.47, 47.04) for the inverse-probability-of-treatment-weighted analysis. For treated patients with a low propensity score, risks of dying were high. Exclusion of patients with a propensity score of <5% yielded comparable odds ratios of approximately 1 for all methods. High levels of nonuniform treatment effect render summary estimates very sensitive to the weighting system explicit or implicit in an adjustment technique. Researchers need to be clear about the population for which an overall treatment estimate is most suitable.