Artificial intelligence is often lauded as a silver bullet for healthcare’s expense and inefficiencies. A new study from the New England Journal of Medicine suggests the opposite may be true, and that in some cases, AI may actually increase the cost of care. It raises the question of what it takes for hospitals and providers to turn AI investment into measurable financial returns.
The study, published last month, evaluated 24 AI systems using 200 de-identified outpatient and emergency department clinical notes. 20 systems were general-purpose LLMs spanning seven developers, while four were specialized medical AI systems. Each system was fed the same clinical presentation as the treating physician and prompted to create its own assessment and treatment plan.
Why Pharma Companies Should Look To Social Media For Better Patient Insight
Healthcare decision-makers, especially at pharmaceutical companies, must prioritize patient input. Data collected from social media listening is a significant step in the right direction.
Researchers compared the care recommendations generated by AI to the plans documented by the physician for each case. Using 2026 Medicare rates, the cost of the AI-generated plan was compared to the real-world physician’s plan, and the results were eyebrow-raising.
The general-purpose LLMs’ plan cost an average of $314 more per visit compared to the physician’s plan. The speciality medical AI systems were no less costly in their suggestions, recommending care that cost 2.7 to 3.5 times as much as the real-world care plan.
The additional costs were mostly driven by diagnostic testing and referrals. In 66.5% of the 200 visits studied, physicians ordered no diagnostic workup, yet every AI system suggested testing in at least some of these cases.
“The care these systems recommended cost substantially more than physicians’ documented care, chiefly because they suggested testing at visits where the physician had ordered none,” the researchers wrote.
Inside an Automated Healthcare Practice: Redesigning Care Around People, Not Paperwork
By reducing administrative burden and redesigning workflows around human needs, it creates space for what matters most: connection between clinicians and patients.
However, the extra recommendations were not necessarily inappropriate. According to the study, Nearly a fifth of AI plans were aligned with clinical guidelines, often catching workups the physician had omitted.
“Excess recommendation is not synonymous with waste,” the researchers cautioned.
For Richard Carter, chief technology officer at Proximie, the findings highlight an important distinction between AI’s potential and its ability to deliver cost-sensitive care in clinical environments.
“The finding [of the study] may be right, but it’s narrow, and the lesson is bigger than cost,” Carter said in an email.
He pointed out that the models only received a written clinical presentation, without the opportunity to question patients or access all the real-time observations available to the treating physician. Carter argues that AI should support clinical decisions rather than independently determining care.
“AI should guide, not decide,” he said.
Carter believes that operational AI can demonstrate results within weeks or months, while clinical AI requires longer-term outcome data before hospitals can confidently claim a return on investment.
The NEJM researchers came to a similar conclusion.
“High-value AI-guided care therefore requires both deliberate safety-aware prompting and clinician oversight to filter residual low-yield recommendations,” they wrote.
As hospitals continue investing in AI, the challenge will be to ensure that recommendations are clinically appropriate, informed by real-world data, and filtered by an experienced clinician in order to yield results that actually improve efficiency and resource utilization while maintaining a high standard of patient care.
Photo: DBenitostock, Getty Images