Cardiologist and researcher Dr. Eric Topol is considered by many to be one of the leading voices contributing to the conversation around technology’s impact on healthcare.
Dr. Topol — who has been serving as founder and director of the Scripps Research Translational Institute for nearly 20 years — recently shared his thoughts on how generative AI is performing in clinical settings. During a keynote address this month at the Radiological Society of North America’s annual meeting in Chicago, he said that while preliminary findings may seem impressive, these results might not hold up in the complex realities of clinical practice.
Several recent studies have found that AI outperforms physicians in clinical tasks, such as differential diagnosis, Dr. Topol pointed out.
Changes in Nurse Staffing Answer Clinician Demands
The ongoing nursing shortage facilitates high turnover rates since nurses know they won’t have difficulties finding new jobs. In order to retain and attract staff, it’s in a facility’s best interest to understand what nurses want.
Some research is even showing that AI outperforms hybrid models, meaning a physician assisted by AI. For example, a study published in JAMA in October showed that OpenAI’s ChatGPT achieved a diagnostic accuracy rate of 90% — while physicians assisted by ChatGPT scored 76% and physicians using only conventional resources scored 74%.
“That isn’t the way it was supposed to work. It was supposed to be that the combined hybrid performance was going to be the best,” Dr. Topol noted.
There are three reasons for this, he added.
Physicians’ bias against automation is one factor that might lead AI to outperform a hybrid model, Dr. Topol noted. Another reason is the fact that physicians still have a limited familiarity with generative AI tools and how to best use them, he stated.
Understanding EGPA: The Role of Eosinophils and Advancements in Treatment Options
FASENRA® (benralizumab) injection, for subcutaneous use, 30 mg is indicated for the treatment of adult patients with eosinophilic granulomatosis with polyangiitis (EGPA). FASENRA provides a treatment option for HCPs to consider when managing this challenging disease.
The third reason is that “these are contrived experiments that are not the real world,” Dr. Topol declared.
Most studies testing generative AI in healthcare are conducted in controlled environments, typically using simulated data that doesn’t come from real patients, he said.
“We wouldn’t want to conclude yet that AI is better than the physician plus AI for these tasks — because these are not real-world medical tasks,” Dr. Topol remarked.
An April paper analyzed more than 500 studies on large language models in healthcare and found that only 5% of them were conducted using real-world patient data, he noted.
“So it should be concluded these are preliminary findings that are not necessarily what we’re going to see when we look at real-world medicine — which is very different than in silico medicine,” Dr. Topol stated.
For most generative AI use cases in the clinical realm, it still remains to be seen whether they can outperform or even match their physician counterparts, he said. This is not true for ambient notetaking models though, Dr. Topol noted.
Hospitals across the country are deploying these tools — which are sold by companies like Abridge, Microsoft, Suki and DeepScribe — in real-life settings, he pointed out.
AI tools for clinical documentation are proving their ability to effectively streamline workflows, increase accuracy, and reduce physicians’ administrative workload by hours per day. In Dr. Topol’s view, these results suggest that the future for generative AI in clinical settings could still be bright.
Photo: Carol Yepes, Getty Images