Evidence-Based Medicine: AI Scribes Actually Work

By now, you’ve surely heard the hype about AI “changing everything” in medicine (and everywhere else). Hopefully, you’ve even tried out some of the technology — most likely an AI scribe. I’ve used an AI scribe in my practice for the past two years and I’m a convert. In fact, I use it for almost 100% of my patient encounters. If you’re not using an AI scribe, I think you’re missing out, but if you don’t want to take my word for it, you’re in luck: 2024 was the first year that high quality, peer-reviewed research articles emerged to help answer questions about the impact of AI scribes in the real world of healthcare. 

“AI scribe” is a popular term for AI-driven ambient listening technology. This technology transforms the conversation between a doctor and a patient into a draft visit note for the electronic health record (EHR). The goal is simple: to help providers save time. Physicians spend countless hours on notes, often after work, which contributes to burnout. AI scribes minimize manual data entry, improve efficiency, and allow doctors to focus more on patient care. Many solutions also integrate directly with EHRs, reducing errors and streamlining workflows.

AI scribes: The research is clear

So, do AI scribes really save time?  One of the most compelling articles on AI scribes was published in the Journal of the American Medical Informatics Association, a flagship journal for Clinical Informatics. Conducted at Stanford, the study followed 45 providers and asked: “Does an AI scribe reduce documentation time?”  The answer was a resounding “yes” — by a median of .57 minutes per note.

That’s about 34 seconds per note. While that number might seem small at first glance, it translates to 11 to 20 minutes daily for a physician seeing 20 to 30 patients. Power users saw even greater time savings, with some reducing documentation by up to 120 minutes per day. This study confirmed that not every provider benefits equally, and this variation in time saved is an area for further research.

In a related paper in the same journal, the providers also reported less burnout, decreased “task load,” and improved usability of their EHR.

Can anyone learn to use an AI scribe?

A major advantage of AI scribes is the reduction of “pajama time,” the hours doctors spend completing notes after work. But is this benefit just for technophiles? 

A 2024 study published in the New England Journal of Medicine Catalyst examined a large-scale rollout of AI scribes at The Permanente Medical Group in California.  This study was massive — in just 10 weeks, AI scribes were deployed to 3,400 physicians who generated 300,000 notes. Even with thousands of providers, the authors found a significant decrease in time spent documenting encounters and a decrease in after-hours documentation (pajama time).

Furthermore, the study measured the quality of the AI scribe notes using a specially developed assessment tool called the PDQI-9. They found that the AI notes were of very high quality, scoring an average of 48 out of 50, rated across 10 key domains like accuracy, usefulness, organization and succinctness.

Can AI really help provider burnout?

Of course, documenting visits is only one of the factors contributing to the epidemic of provider burnout. Can the time savings and other benefits of AI Scribes really improve the well-being of providers?

The emerging evidence-based answer appears to be yes. In an article in Applied Clinical Informatics, the use of AI scribes was found to significantly reduce provider burnout (as measured with a validated tool). It was also found to improve interpersonal disengagement scores and enhance professional fulfillment, suggesting that the promise of improved physician work life is quite real.

Challenges on the road ahead

Despite the promising benefits, AI scribes face several challenges. They can misinterpret speech, medical terminology, or context, necessitating human oversight. Some clinicians are hesitant to trust AI-generated notes due to concerns about accuracy or bias. Other barriers include adoption costs, training, provider hesitancy, integration challenges with EHR systems, and ethical and privacy considerations. 

However, these hurdles are not insurmountable. AI scribes are rapidly evolving, and in my experience, they save time daily, ensure key details are not missed, and reduce the stress of after-hours documentation.

Final prognosis: AI scribes are here to stay

AI scribes represent a significant advancement in healthcare technology, and peer-reviewed evidence is building in support of their effectiveness. They offer a practical solution to the documentation burden that plagues many healthcare providers. By reducing the time spent on administrative tasks, AI scribes allow physicians to focus more on what truly matters — patient care. As the technology continues to evolve and improve, its adoption will likely become more widespread, further enhancing its benefits.    

AI scribes aren’t perfect, but as the saying goes — don’t let perfect be the enemy of good. Many providers assume that AI can’t help them after only giving it a half-hearted try. I have personally helped convert numerous AI nay-sayers into ambient listening fans by just handholding them through a few encounters. 

I strongly encourage you to experience the impact of an AI scribe on your practice by performing a little research of your own — a trial run, preferably with a cutting-edge AI tool that offers full EHR integration. After a few days, simply ask yourself, “Is this saving me time or making my day better?”    

I have a hypothesis about what your findings will be.

Photo: Filograph, Getty Images

This post appears through the MedCity Influencers program. Anyone can publish their perspective on business and innovation in healthcare on MedCity News through MedCity Influencers. Click here to find out how.