In a recently published article in MedCity News, Fran Rosch, the CEO of Imprivata, a cybersecurity firm focused on credentialing and identity security across sectors, suggested that the best approach to interacting with AI agents should be transactional and self-limiting, with regular reminders that these agents are not to be trusted. He then drew an uncomfortable parallel between AI agents and contract nurses and underscored how both groups should be treated similarly in the healthcare sector.
However, the reality is that the Imprivata CEO is missing the point entirely. From the standpoint of a cybersecurity company, what he is saying makes sense. Treating AI agents like contract nurses and minimizing their access to critical patient information certainly reduces potential risk. But, it also minimizes key opportunities.
Extending Rosch’s analogy, if we compare agentic AI to a nurse, we need to remember that, just like with a nurse, AI benefits from learning. Allowing AI to work on a problem longitudinally allows it to more effectively adjust to the specific needs of the client, learn necessary behaviors and workflows, create more sustained engagement, and ultimately improve the processes leading to a better ROI.
Automation Didn’t Dehumanize Care. Bad Systems Did.
We have the opportunity to design systems that give people greater capacity for the work where human presence matters most: listening, understanding, reassuring, deciding, and caring.
Perhaps more importantly, organizations do not like working with temps. Continuity matters — both with human employees and technology. This is especially true in healthcare, where relationships and trust are fundamental to engagement. It is essential, based on my professional experience, to build a virtual healthcare companion that interacts with patients over time. For that model to work, patients need to develop trust not only in the healthcare organization, but also in the technology they are interacting with — and, this trust can be challenging to build, particularly for certain vulnerable patient demographics. A patient who sees a familiar AI companion that understands their history, preferences, needs, and patterns is likely to have a very different, and much more positive, relationship with that technology than a patient who encounters a new AI agent every time a task needs to be completed.
Trust in technology does not happen with a snap of a finger or because someone tells a user to trust it; it is built over time. An “adopt and drop” model may minimize risky exposure from a cybersecurity perspective, but it also creates a cycle of new introductions, new permissions, new workflows, and new opportunities for something to go wrong. Staying with the same AI, monitoring it, learning from its behavior, and gradually expanding what it is trusted to do can actually create a stronger security and trust environment.
There is also a patient-engagement argument that cybersecurity thinking alone misses. In healthcare, the goal is not simply to complete a transaction safely. Patients need to engage, share information, follow through with recommendations, and remain connected to their care. All of this depends on trust. If an AI system is going to remind a patient about an appointment, help them navigate a treatment plan, answer questions, or follow up after a visit, the patient needs to view and accept it as a consistent and trusted part of their care experience.
Just like with any other employee, the more time we spend working with AI, the better we understand each other. Perhaps the question should not be how quickly we can cut an AI agent off, but how we can safely allow it to earn our trust. Restricting AI to the limited, self-contained interactions Rosch describes may be safer, but it risks throwing the proverbial baby out with the bathwater. In doing so, we are sacrificing the far greater upside potential of developing a trusted and trainable AI agent in the name of cybersecurity, or perceived cybersecurity risks. The trick is not to make AI agents safe by isolating them operationally, but instead to make them trustworthy enough to operate safely in the real world.
Why Pharma Companies Should Look To Social Media For Better Patient Insight
Healthcare decision-makers, especially at pharmaceutical companies, must prioritize patient input. Data collected from social media listening is a significant step in the right direction.
Photo: Issarawat Tattong, Getty Images
Dr. Gennady Gekht is the Chief Clinical Officer of Hygia and a Mayo Clinic–trained, board-certified neurologist with more than 20 years of experience across neurology and digital health. He was among the founding physicians involved in the clinical development and early trials of Nevro’s implantable spinal cord stimulation platform and has contributed to research and physician training at Stryker and Abbott.
He holds an MBA from Emory University and combines his clinical and business expertise to advance healthcare innovation and the effective and safe use of artificial intelligence. At Hygia, he focuses on how technology can support patients with chronic conditions between visits, strengthen care continuity, and help care teams better understand and address the barriers patients face in following their care plans. Dr. Gekht is a frequent speaker at scientific conferences and educational forums in the United States and internationally.
This post appears through the MedCity Influencers program. Anyone can publish their perspective on business and innovation in healthcare on MedCity News through MedCity Influencers. Click here to find out how.