The Patient-Centered Medical Home and the Challenge of Evaluating Complex Interventions

The study by Reddy and colleagues1 reports on an analysis of the Patient Aligned Care Team (PACT) initiative, a patient-centered medical home (PCMH) initiative from the US Department of Veterans Affairs (VA) and one of the largest PCMH initiatives in the country. Involving more than 1 million patients during 4 years (2012-2015), the study tapped into the extensive databases of the VA, including results from surveys of patients and staff specifically designed to capture elements of PCMH. Reddy et al1 examined the association of longitudinal changes in PCMH implementation with 3 high-cost health care utilization outcomes: emergency department (ED) visits, hospitalizations for ambulatory care–sensitive conditions (ACSCs), and all-cause hospitalizations. They found no consistent association of more robust PCMH implementation with better outcomes. While there has been considerable enthusiasm about the potential for PCMH to revolutionize primary care in the United States, PCMH is more of a collection of core principles than a set of specific interventions. Many different interventions may address the various principles of PCMH, with the result that PCMH initiatives can look very different across the organizations attempting them. Two reviews from 2017 have confirmed this variation,2,3 and they also found limited evidence of associations with most clinical and financial outcomes, with more consistent findings involving small to moderate improvement with selected prevention and chronic disease management measures. The complexity of the intervention being attempted in the VA PCMH initiative is evidenced by the main measure for its implementation, the PACT implementation progress index (Pi2) score. The Pi2 score consists of 53 different elements, covering 8 domains, which are drawn from patient surveys, staff surveys, and measures of clinic operations.1 This score has been used to assess many outcomes, covering all the dimensions of health care quality as well as staff satisfaction and burnout. Given that a basic principle of scientific investigation is to control as many variables as possible, ideally isolating a single independent variable to study its effect or associations, it should not be surprising that the findings of system-level studies that include multiple independent variables and multiple outcomes of PCMH interventions have included mixed results. There are many possible explanations for this outcome. Perhaps PCMH is not effective in reducing costs. However, it is also possible that out of the many elements of PACT, those that can reduce ED visits or hospitalizations were not effectively implemented or were implemented to a similar degree in the 2 periods studied. This study is of particular interest because the same authors had previously reported impressive differences in outcomes in a cross-sectional analysis between clinics with high Pi2 scores for PCMH implementation vs those with low Pi2 scores.4 They found that the practices that received the highest Pi2 scores had lower hospitalization rates for ACSC and lower ED use. In contrast, in this study that examined a 4-year implementation period, Reddy et al1 did not find any consistent associations of longitudinal differences between PCMH implementation and rates of all-cause hospitalization, ACSC hospitalizations, and ED visits. Why the difference between the 2 studies performed by these authors (cross-sectional4 vs longitudinal1), and why has it been so difficult to show benefit from PCMH implementation? First, implementing complex system changes, such as PCMH, in health care is not easy. Possibly, highfunctioning clinics that already perform well on desired outcomes may be more successful in executing changes requested by their parent organization. Cross-sectional analyses may find associations of successful implementation with high performance but these associations may not be + Related article