Patients are increasingly walking into exam rooms having already consulted an AI chatbot about their symptoms — and sometimes trusting it as much as they trust their doctor.
But when a chatbot gets it wrong and a patient gets hurt, who bears the responsibility? According to Meghan O’Connor, health law partner at Quarles & Brady and co-chair of the firm’s AI team, the answer is still being written.
“This is one of the most significant open questions in healthcare law right now, and the honest answer is that liability allocation is going to be highly fact-dependent and will likely take years of litigation to clarify,” she remarked.
The Healthcare Payments Industry Faces a Perception Challenge
Healthcare payments are no longer about transactions. It’s about controlling the revenue cycle. And increasingly, that control is not sitting with ISOs.
O’Connor broke the liability question into three layers: the AI developer, the patient and the provider.
AI companies often lean on disclaimers stating their outputs aren’t medical advice, but those disclaimers might not hold up if a chatbot was marketed in a way that encouraged patients to treat its answers as diagnostic guidance, she explained.
As for patients, O’Connor says courts are unlikely to pin much blame on them for trusting a tool that presents itself as authoritative.
Providers have to walk a finer line, though. Once a patient discloses they’ve been relying on inaccurate AI advice, the provider’s failure to correct it could become a malpractice issue. O’Connor said this standard-of-care issue boils down to what a reasonable provider would have done with that same information.
Inside an Automated Healthcare Practice: Redesigning Care Around People, Not Paperwork
By reducing administrative burden and redesigning workflows around human needs, it creates space for what matters most: connection between clinicians and patients.
“My practical recommendation for providers: correct misinformation, document the conversation and treat AI-sourced information the same way you’d treat any patient-reported information that conflicts with clinical judgment — address it directly and note that you did,” she stated.
Once a patient starts discussing AI-generated information during a clinical encounter, that information becomes part of the clinical picture the same way as a patient reporting web-browsing results or advice from their friend, O’Connor noted.
She said silence is rarely the safer option.
“Think of it this way: if a patient told you they read on an internet forum that drinking pickle juice would cure their glaucoma, no reasonable ophthalmologist would say nothing. AI-sourced misinformation should be treated the same way. The source of the bad information does not change the provider’s reasonable duty to the patient once it’s been raised in the context of care,” O’Connor declared.
The law hasn’t caught up to AI chatbots yet, but her bottom-line advice for providers is to engage, not ignore. It’s a small investment of time that she said could matter a great deal down the line.
Photo: Witthaya Prasongsin, Getty Images