Generative AI is rapidly changing healthcare with solutions bringing efficiency to clinical documentation, decision support, and patient communication to name just a few applications. These new solutions tools are helping to reduce doctor’s administrative burden (AKA “pajama time”), increase the speed to clinical decisions, and improve patient engagement. A survey by Menlo Park found that 22% of healthcare organizations have implemented AI tools.
However, like with every major technological leap, there are challenges. The main concern is not just how GenAI can affect change, but how to implement it effectively without harming trust or safety.
As with other disruptive technologies, the current GenAI conversation is saturated with claims of impact. Yet, technology alone won’t save healthcare. The real work is tackling deeper systemic issues from misaligned incentives to fragmented workflows, as well as the ever-present risk of undesirable outcomes.
The Power of One: Redefining Healthcare with an AI-Driven Unified Platform
In a landscape where complexity has long been the norm, the power of one lies not just in unification, but in intelligence and automation.
GenAI has real promise, but it’s also bringing real risks. It is not about whether we should use it but how we can use it responsibly and with positive outcomes.
The rise of shadow AI: Innovation outpacing oversight
As GenAI tools proliferate, a new risk emerges: “shadow AI” — the use of generative AI by clinicians outside institutional oversight. This isn’t a theoretical concern. When innovation outpaces governance, it opens the door to misinformation, “hallucinations,” and decisions that may look correct but are fundamentally flawed. For example, a question about treatment for an acute complicated urinary tract infection treated in the outpatient setting of care would return fluoroquinolones. And that is a correct answer IF the patient isn’t pregnant — as fluoroquinolones should be avoided in pregnancy due to risks to the fetus.
GenAI in clinical decision making needs to do more than just feed answers. It needs to clarify and frame what and who the provider is asking about to ensure that the appropriate level of clinical nuance is represented.
Enhanced Direct Enrollment: Georgia Access and the Power of Partnership
Across the nation, state-based health insurance exchanges are searching for ways to better serve their communities while making enrollment easier, more personal, and more accessible for everyone.
Healthcare’s history is littered with examples of technology adoption outpacing readiness. We’ve seen it with EHRs, with population health tools, and now with GenAI. The lesson is clear: experimentation does not equal preparedness. Responsible leaders must ask not just “can we deploy this?” but “should we, and, if so, under what guardrails?”
The governance gap: Why oversight matters
The governance gap exists between the value that GenAI can deliver and how it aligns and adheres to the policies and guidelines that are intrinsic to it or regulated. If that gap is not given oversight the impact can be detrimental to care, potentially adding to disparities and eroding the trust that is essential to healthcare.
Implementing governance is not a zero-sum action. It will not restrain innovation or impede deployment. Having clear guidance can empower the creation of new solutions due to clear policies that give clarity around how applications and tools should be used, what data they should leverage and what accountability looks like. Effective governance also means ensuring the infrastructure is secure, monitored and enables detection of unsafe or unsanctioned use. AI tools must be embedded in secure, monitored environments, with real-time detection of unauthorized or unsafe use including:
- Ensuring clinicians are empowered through training not just on the value of its use but the limits and risk of its use.
- An understood system of monitoring, feedback, and adaptation as technology and clinical practice that is both ongoing and changes to include new tools and applications.
Separating the hype and the value
The path forward is clear: shift from hype to value. The real test for GenAI is not in its novelty, but in its ability to deliver measurable improvements for clinicians and patients. That means demanding tangible ROI — reduced administrative burden, improved outcomes, and enhanced patient experience. It also means being honest about what GenAI can’t do, and where human judgment remains irreplaceable.
Healthcare leaders must insist on continuous and rigorous evaluation, transparent and regular reporting, and a focus on improvement. The goal isn’t just to adopt GenAI, but to integrate it in ways that support clinicians, empower patients, and advance health equity.
Leading through the GenAI revolution
The GenAI revolution is here, but its success won’t be based on the technology we select and use but on how we implement it. Will we allow shadow AI to flourish, will we close the governance gap and ensure that innovation is our most important role?
The answer lies in strong leadership. Healthcare leaders must seek tools grounded in evidence, identify and work to a clear policy, and are animated by a commitment to patient safety and trust. If we get this right, GenAI won’t just transform healthcare workflows; it will help us build a healthcare system that is more efficient, equitable, and resilient for all.
Photo: francescoch, Getty Images
Holly Urban, MD, MBA has extensive experience in healthcare technology and believes in the power of evidence-based content to transform EHRs beyond transactional systems into tools that allow clinicians to provide improved patient outcomes. After practicing as a primary care pediatrician, Dr. Urban worked for several EHR technology and evidence-based content companies, and has served in healthcare IT leadership roles for over fifteen years. Before joining Wolters Kluwer Health, she served as CMIO at Oracle Cerner, Director of Product Management at MCG Health, and VP of Product Management at McKesson Horizon Clinicals.
This post appears through the MedCity Influencers program. Anyone can publish their perspective on business and innovation in healthcare on MedCity News through MedCity Influencers. Click here to find out how.