Premium
How study design affects outcomes in comparisons of therapy. I: Medical
Author(s) -
Colditz Graham A.,
Miller James N.,
Mosteller Frederick
Publication year - 1989
Publication title -
statistics in medicine
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.996
H-Index - 183
eISSN - 1097-0258
pISSN - 0277-6715
DOI - 10.1002/sim.4780080408
Subject(s) - statistic , randomized controlled trial , research design , medicine , mann–whitney u test , statistics , completely randomized design , random assignment , randomized experiment , clinical trial , sample size determination , sequential analysis , treatment effect , mathematics , traditional medicine
We analysed 113 reports published in 1980 in a sample of medical journals to relate features of study design to the magnitude of gains attributed to new therapies over old. Overall we rated 87 per cent of new therapies as improvements over standard therapies. The mean gain (measured by the Mann—Whitney statistic) was relatively constant across study designs, except for non‐randomized controlled trials with sequential assignment to therapy, which showed a significantly higher likelihood that a patient would do better on the innovation than on standard therapy ( p = 0.004). Randomized controlled trials that did not use a double‐blind design had a higher likelihood of showing a gain for the innovation than did double‐blind trials ( p = 0.02). Any evaluation of an innovation may include both bias and the true efficacy of the new therapy, therefore we may consider making adjustments for the average bias associated with a study design. When interpreting an evaluation of a new therapy, readers should consider the impact of the following average adjustments to the Mann—Whitney statistic: for trials with non‐random sequential assignment a decrease of 0.15, for non‐double‐blind randomized controlled trials a decrease of 0.11.