Accountability, secrecy, and innovation in AI-enabled clinical decision software

This article employs analytical and empirical tools to dissect the complex relationship between secrecy, accountability, and innovation incentives in clinical decision software enabled by machine learning (ML-CD). Although secrecy can provide incentives for innovation, it can also diminish the ability of third parties to adjudicate risk and benefit responsibly. Our first aim is descriptive. We address how the interrelated regimes of intellectual property law, Food and Drug Administration (FDA) regulation, and tort liability are currently shaping information flow and innovation incentives. We find that developers regard secrecy over training data and details of the trained model as central to competitive advantage. Meanwhile, neither FDA nor adopters are currently asking for these types of details. In addition, in some cases, it is not clear whether developers are being asked to provide rigorous evidence of performance. FDA, Congress, developers, and adopters could all do more to promote information flow, particularly as ML-CD models move into areas of higher risk. We provide specific suggestions for how FDA regulation, patent law, and tort liability could be tweaked to improve information flow without sacrificing innovation incentives.