Health IT, Artificial Intelligence

Consumers don’t fully trust healthcare chatbots, study finds

A new report from UserTesting reveals how 500 consumers evaluated the following five healthcare chatbot apps: Ada, HealthTap, Mediktor, Your.MD and Symptomate.

AI, artificial intelligence, chatbot, bot

A new report from UserTesting, a company offering an on-demand human insight platform, found healthcare chatbots aren’t quite meeting users’ expectations, particularly when it comes to trust, handling complex conditions and usability.

UserTesting ran a study of five healthcare chatbot apps: Ada, HealthTap, Mediktor, Your.MD and Symptomate.

Five hundred consumers evaluated the apps on ease of use, speed, credibility, aesthetics and delight. Participants were given a list of symptoms for the common cold and for food poisoning. They were asked to use the apps to get a diagnosis, then answer a list of questions about their experience.

Based on the results, consumers don’t completely trust the chatbots. They were wary of relying on unfamiliar brands and had concerns about HIPAA compliance. As one participant stated: “I was hoping to see a notice that this app would comply with HIPAA standards.” Others said they would prefer to conduct follow-up research before they could trust the diagnosis and information provided. Additionally, some participants complained of spelling errors and links to content that seemed untrustworthy.

On a scale of 0-100, Ada got the highest credibility score of 82.7. Symptomate received the lowest credibility score, coming in at 41.4.

Another takeaway from the study is that chatbots have room for improvement when it comes to complex conditions. When users tried to diagnose food poisoning, the chatbot experience became less intuitive and more overwhelming. But as the report notes, it’s key to remember that “because the symptoms participants were asked to seek diagnosis for were hypothetical, their responses could be attributed to how they perceived those symptoms, rather than how they’d actually feel if they had them.”

Mediktor earned the highest complex diagnosis score of 73.8. Ada was a close second at 73.7, while HealthTap had the lowest score (43.1).

Basic usability is also a challenge when it comes to these apps. For instance, some participants had loading issues with HealthTap and had to completely exit the app and restart their diagnosis process.

Overall, the report notes that although healthcare chatbots hold promise, there’s still a long way to go before they win over consumers. Still, after the study, the majority of participants (73 percent) felt the tools were helpful. Twelve percent said the chatbots were not helpful, and 15 percent remained neutral.

Photo: Jull1491, Getty Images

Shares1
Shares1