Discovering effect modification in an observational study of surgical mortality at hospitals with superior nursing

Summary There is effect modification if the magnitude or stability of a treatment effect varies systematically with the level of an observed covariate. A larger or more stable treatment effect is typically less sensitive to bias from unmeasured covariates, so it is important to recognize effect modification when it is present. We illustrate a recent proposal for conducting a sensitivity analysis that empirically discovers effect modification by exploratory methods but controls the familywise error rate in discovered groups. The example concerns a study of mortality and use of intensive care units in 23715 matched pairs of two Medicare patients, one of whom underwent surgery at a hospital that had been identified for superior nursing; the other at a conventional hospital. The pairs were matched exactly for 130 four‐digit ninth international classification of diseases surgical procedure codes and balanced 172 observed covariates. The pairs were then split into five groups of pairs by the classification and regression trees method in its effort to locate effect modification. The evidence of a beneficial effect of magnet hospitals on mortality is least sensitive to unmeasured biases in a large group of patients undergoing quite serious surgical procedures, but in the absence of other life‐threatening conditions, such as a comorbidity of congestive heart failure or an emergency admission leading to surgery.