FDA Official Believes GenAI-enabled Tools Will Need to Be Assessed Like a Physician

At an in-depth panel discussion Tuesday at The MedTech Conference hosted by the industry group AdvaMed in Boston, Rick Abramson said something very interesting on the future of AI regulation.

Abramson, a Harvard-trained radiologist and the director of FDA’s Digital Health Center of Excellence, said generative AI tools will perhaps need to be evaluated similarly to a physician who is routinely evaluated for competency. This would be a departure from how the agency has traditionally evaluated the safety and efficacy of software as a medical device. Artificial intelligence, after all, is souped-up software.

“We’ve been looking at software as a medical device for 20 years. We’re used to a paradigm where you have a piece of software — you can put in X, and it spits out Y — in a nutshell,” Abramson explained. “I come from the radiology world [and with] AI devices, you put in a chest X-ray, it’ll give you some kind of signal, like a binary signal. ‘Yes, no, there is a pneumothorax.’ Classic example, right? Well, that’s pretty straightforward from an evidentiary standpoint.”

And now with GenAI tools, we are in a world where the input is limitless text as is the output, which means the paradigm for evaluating safety and efficacy also has to change.

“Well, now we’re in a situation where we have this unbounded evaluation space. What are we going to do for that? We don’t have the methods for that type of evidentiary proof,” he declared.

In August, the FDA put out a detailed discussion paper to elicit feedback from the industry about how these new GenAI-enabled medical devices should be regulated. Abramson referred to it as he explained the agency’s thinking.

“We ended up putting out this idea that maybe these devices are starting to behave a little bit more like me as a physician than a heart valve or a defibrillator,” he said. “I went through a system where, over time, I was evaluated on my knowledge and my clinical judgment and my communication ability and my ability to adhere to certain safety boundaries. I’ve never been given a test where I’m submitted to every possible chest X-ray I could ever encounter over the course of my career. This is not practical or possible.”

The comment period for the paper ended on Monday and there are more than 300 comments that the agency has received. So the obvious question came from the moderator Geeta Pamidimukkala, AdvaMed’s vice president of technology and regulatory affairs: After this discussion paper and all the comments, what will we see next from the agency?

“So let me just say very directly, we intend to go to policy. We intend to draft policy. This is not just an academic paper. It’s not just a thought exercise,” Abramson said emphatically. “I’m not a political appointee, but I can tell you that this administration has made this policy development a priority and I’m talking about at the level of the [HHS] Secretary and the White House.”

And when might we expect a draft guidance on the policy? Abramson was intentionally vague on the timeline. He cited the voluminous responses received and the desire to read through all of them thoughtfully before putting out draft guidance.

“Look, we intend to read the comments. We’re not just going to run them through an AI engine,” he said to audience laughter. “We’re going to read and we’re going to synthesize. We want to be very thoughtful about this because this is really important. So I would say not on the course of weeks, but certainly not on the course of years. I would say on the course of months, expect draft policy guidance to be issued and then obviously there’ll be another opportunity for stakeholder comment that we look forward to.”

It is interesting how the agency’s thought process is evolving and how Abramson described the idea of evaluating GenAI-enabled devices like flesh-and-blood physicians. One group that has commented on the agency’s discussion paper is the American Medical Association, whose CEO, John Whyte, sent a letter to Abramson just last week. The letter contained the physician group’s recommendations on what the agency should consider as it develops a robust regulatory framework for regulating generative AI and agentic AI tools. One issue that Whyte addresses in the letter is “practicality,” something that Abramson said is leading the agency to modify some of its thinking on testing the safety and efficacy of these new tools.

“Neither the probabilistic nature of GenAI outputs nor the impossibility of testing every conceivable scenario should diffuse that accountability across clinicians, patients, deploying institutions, foundation-model developers, or other participants in the ecosystem,” Whyte wrote in the letter.

Photo: steved_np3, Getty Images