Why Agentic AI Works in Hospitals When You Strip It Down

Imagine an agentic AI solution that plans and executes multi-step tasks like triaging referrals, drafting notes, flagging deteriorating patients, and surfacing care recommendations. After the flawless demo, the committee approves what is supposed to be the answer to operational inefficiencies in healthcare. Fast forward six months: the system formally operates, but it no longer influences any real clinical decisions.

Many hospitals are now trying to use AI to automate processes and free up staff time, running simultaneous pilots to perform as many tasks as possible. This quickly leads to pilot sprawl — a state where organizations can no longer even say which agents are still in active use.

As one widely cited MIT analysis has shown, 95% of enterprise AI pilots fail to deliver measurable ROI, and I would add that they fail predictably. From my experience, I would distinguish the four most common scenarios:

• The Big Bang: an attempt at a long, all-department transformation, so by the time anything goes live, the requirements have already changed, and the budget was spent before a single user has seen value.
• Pilot Purgatory: a project that looks impressive in a controlled environment (demo) but cannot pass the reality check, working with human users, actual workflows, and unexpected interruptions.
• Technology-First: the platform gets built before the problem it should solve is clearly defined, so the team ends up searching for at least some use case after the investment is already made.
• Governance Paralysis: long months spent designing the perfect oversight model while the initiative quietly loses momentum, ownership, and executive support.

In fact, all four scenarios share the same fundamental mistake: the system is designed to look intelligent, convincing, even flashy, but not as one you can trust in a real healthcare workflow.

What especially shines in a demo often arrives as opaque in practice. Each new agent and each additional step creates another space where something can go wrong.

An annoying glitch in a consumer app translates to delayed critical care or missed clinical alerts in a hospital. The human is never genuinely in the loop, so staff make the safest and most obvious decision and stop using the system they cannot fully rely on.

Even if the model is robust, an agent operating outside the chart creates too much friction for daily use. Recently, Jonathan Bush argued in MedCity News that orchestration layers must be built above the EHR. While his diagnosis of EHR limitations is spot on, the proposed solution misses the reality of clinical workflows. An agent must live directly inside the patient chart via standard integrations. If it exists as a separate overlay, it will simply be abandoned.

After a decade of working on EHR and HL7 FHIR integration projects, I see this exact friction more often than almost any model problem. For example, the clinician has to leave the record, open a second tool, and then come back just to act on the recommendation. A strong triage engine can be abandoned for exactly that reason.

How do we handle all these issues? The fix is not a better model, but the right sequence. First comes assessment, then the pilot, then change management, and only then scaling. Between these phases there must be a real decision gate, and you should be ready to stop if there is no good reason to go further.

• Assessment: The assessment should be brief — about four weeks. We need it to reduce risks before building anything. The potential use cases must be evaluated based on such criteria as value, feasibility, and personal relevance. Value shows the return you get. Feasibility shows whether it is achievable with the available data and infrastructure. And personal relevance is about whether the solution is useful to a particular person in their daily work. If the assessment is honest, the result is a readiness scorecard and a go/no-go call before the money is spent.
  
• Pilot: It is not just a slideshow, but a real demonstrator that works with real users and real data in production, typically over twelve weeks. To pass the reality check and avoid getting lost in pilot purgatory, it must do one clearly defined and practical thing. If you cannot explain to a floor nurse in one sentence what exactly this AI agent does, you do not have one project but several. And this agent must surface in the chart clinicians already work in, not in a separate tool they have to open and use on its own. The system’s rationale must be clear — the specialist using it must understand why a particular case or patient surfaced. And of course, the human must be in the loop, not for show. The system can work and make sense only when the agent surfaces recommendations while a specialist makes the final clinical decision. On one documentation project built this way, follow-up documentation time fell by about a third and compliance with scheduled check-ins doubled. This aligns with broader industry validation; for instance, the AHA Market Scan noted a comparable 27% reduction in documentation time at Intermountain Health.
  
• Change Management : Change management is probably the most underrated step that many teams skip. If no one wants to use the agent, it doesn’t matter how technically excellent it is. So, before you scale, you need not only to train the team how to use the solution, but also build it into daily processes and see whether it has actually caught on. You can see it in very simple signals — how often a nurse accepts or declines a suggestion and what share of recommendations she opens versus ignores. Those metrics are both proof of adoption and a map of what still needs improvement.
  
• Scale: Finally, it makes sense to talk about scaling. And not because “everyone liked the pilot,” but because it met specific predefined criteria and showed real results, such as time saved per shift, error rate, and task compliance. If something is still missing in the pilot, you iterate or stop. If the demonstrator clears the bar, you earn a slightly broader next scope, one use case at a time. That’s how you can see value by week twelve, rather than postponing it for a year and a half. Evidence first, hype never. Federal transparency rules, such as the HTI-1 final rule, are already moving this way by demanding strict algorithm transparency in certified health IT.  

In healthcare, the systems that scale are not the most ambitious ones, but the ones that are easiest to trust, govern, and use in real workflows. 

The agents that succeed tend to do one narrow task clearly, visibly, and with a human still making the decision.

And the hospitals that will pull ahead over the next couple of years are not the ones running the most impressive pilots. They are the ones running the fewest — and finishing them.

Photo: akinbostanci, Getty Images

This post appears through the MedCity Influencers program. Anyone can publish their perspective on business and innovation in healthcare on MedCity News through MedCity Influencers. Click here to find out how.