Ryan Bioethicist in Residence

 “Accountability and Non-Domination in the Use of AI Systems in Medicine: Validation vs. Explainability”

Alex J. London, Ph.D.

Breakthroughs in machine learning are enabling the use of artificial intelligence (AI) to perform a wide range of diagnostic and predictive tasks in medicine. This prospect has prompted utopian hype, as well as dystopian hysteria, dramatizing the importance of ensuring that systems involved in life-and-death decisions merit public trust.

Essential to securing such trust are clear practices and procedures to ensure accountability and respect for the freedom of stakeholders. A common proposal for achieving these goals imposes requirements like explainability or interpretability that seek, in different ways, to lay out the operation of such systems to human inspection. Because the most powerful AI systems are often opaque “black-boxes,” these requirements may be purchased at the price of reduced predictive accuracy.

In this talk, Professor London will argue that such requirements are misguided in domains—such as medicine—where our knowledge of fundamental causal relationships is precarious and under-developed. Instead, he will argue that we should promote trust and accountability by clearly defining the tasks such systems can perform, the conditions necessary to ensure acceptable system performance, and rigorously validating their accuracy under those well-defined conditions.