FDA Regulation of Predictive Clinical Decision‐Support Tools: What Does It Mean for Hospitals?

Recent experiences in the transportation industry highlight the importance of getting right the regulation of decision-support systems in high-stakes environments. Two tragic plane crashes resulted in 346 deaths and were deemed, in part, to be related to a cockpit alert system that overwhelmed pilots with multiple notifications.1 Similarly, a driverless car struck and killed a pedestrian in the street, in part because the car was not programmed to look for humans outside of a crosswalk.2 These two bellwether events offer poignant lessons for the healthcare industry in which human lives also depend on decision-support systems. Clinical decision-support (CDS) systems are computerized applications, often embedded in an electronic health record (EHR), that provide information to clinicians to inform care. Although CDS systems have been used for many years,3 they have never been subjected to any enforcement of formal testing requirements. However, a draft guidance document released in 2019 from the Food and Drug Administration (FDA) outlined new directions for the regulation of CDS systems.4 Although the FDA has thus far focused regulatory efforts on predictive systems developed by private manufacturers,5,6 this new document provides examples of software that would require regulation for CDS systems that hospitals are already using. Thus, this new guidance raises critical questions—will hospitals themselves be evaluated like private manufacturers, be exempted from federal regulation, or require their own specialized regulation? The FDA has not yet clarified its approach to hospitals or hospital-developed CDS systems, which leaves open numerous possibilities in a rapidly evolving regulatory environment. Although the FDA has officially regulated CDS systems under section 201(h) of the Federal Food, Drug, and Cosmetic Act (1938), only recently has the FDA begun to sketch the shape of its regulatory efforts. This trend to actually regulate CDS systems began with the 21st Century Cures Act (2016) that amended the definition of software systems that qualify as medical devices and outlined criteria under which a system may be exempt from FDA oversight. For example, regulation would not apply to systems that support “population health” or a “healthy lifestyle” or to ones that qualify as “electronic patient records” as long as they do not “interpret or analyze” data within them.7 Following the rapid proliferation of many machine learning and other predictive technologies with medical applications, the FDA began the voluntary Digital Health Software Precertification (Pre-Cert) Program in 2017. Through this program, the FDA selected nine companies from more than 100 applicants and certified them across five domains of excellence. Notably, the Pre-Cert Program currently allows for certification of software manufacturers themselves and does not approve or test actual software devices directly. This regulatory pathway will eventually allow manufacturers to apply under a modified premarket review process for individual software as a medical device (SaMD) that use artificial intelligence (AI) and machine learning. In the meantime, however, many hospitals have developed and deployed their own predictive CDS systems that cross the boundaries into the FDA’s purview and, indeed, do “interpret or analyze” data for real-time EHR alerts, population health management, and other applications. Regulatory oversight for hospitals could provide quality or safety standards where currently there are none. However, such regulations could also interfere with existing local care practices, hinder rapid development of new CDS systems, and may be perceived as interfering in hospital operations. With the current enthusiasm for AI-based technologies and the concurrent lack of evidence to suggest their effectiveness in practice, regulation could also prompt necessary scrutiny of potential harms of CDS systems, an area with even less evidence. At the same time, CDS developers—private or hospital based—may be able to avoid regulation for some devices with well-placed disclaimers about the intended use of the CDS, one of the FDA criteria for determining the degree of oversight. If the FDA were to regulate hospitals or hospital-developed CDS systems, there are several unanswered questions to consider so that such regulations have their intended impact. First, does the FDA intend to regulate hospitals and hospital-developed software at all? The framework for determining whether a CDS system will be regulated depends on the severity of the clinical scenario, the ability to independently evaluate the model output, and the intended user (Table). Notably, many types of CDS systems that would require regulation under this framework are already commonplace. For example, the FDA intends to regulate software that “identifies patients who may exhibit signs of opioid addiction,” a scenario similar to prediction models already developed at academic hospitals.8 The FDA also plans to regulate a software device Corresponding Author: Gary E Weissman, MD, MSHP; Email: gary.weissman@ pennmedicine.upenn.edu; Telephone: 215-746-2887; Twitter: @garyweissman.