Voice assistants, such as Alexa and Siri, have increasingly been expanding their capabilities in recent years, with many being so bold as to offer users health advice.  Research from Stanford explores just how reliable they are.

For many years, people have been turning to “Dr. Google” to help them with their health queries, with sufficiently mixed results that medical campaigners in Belgium specifically asked people to contact professionals rather than conduct self-diagnoses online.  With voice assistants, the prospect of misinformation being shared increases further still.

“As voice assistants become more ubiquitous, we need to know that they are reliable sources of information—especially when it comes to important public health matters,” the researchers say.

Testing reliability

The researchers put various voice assistants to the test in response to a range of common questions around cancer screening.  The study found that many of the assistants were not really able to provide reliable answers, while others weren’t able to provide an answer at all.

“These results suggest there are opportunities for technology companies to work closely with health care guideline developers and health care professionals to standardize their voice assistants’ responses to important health-related questions,” the researchers explain.

Cancer screenings were chosen as a topic because of the importance of securing an early diagnosis.  Screening rates also fell during the pandemic, with doctors delaying a lot of non-essential care, which meant many people went online for information.

A mixed bag

Volunteers asked various voice assistants whether they should be screened for one of up to 11 different forms of cancer.  The response from the voice assistants was mixed, with Alexa often saying they don’t know, whereas Siri would simply provide some web pages to look at.

What’s more, the resources the assistants did send people to were only reliable around 60-70% of the time, with Cortana’s accuracy dropping to just 45% when giving a verbal response.  The results illustrate the considerable room for improvement needed before these tools can provide reliable support to people looking for medical information.

While the tools would often send people to reliable sources, there were also numerous instances of unreputable sources being shared too.  This is especially concerning if the systems in part base the results they give people on the past search patterns of those users.  As the pandemic showed, people can easily fall down a rabbit hole of misinformation that is hard to clamber out of.

“Individuals need to receive accurate information from reputable sources when it comes to matters of public health,” the researchers conclude. “This is important now more than ever, given the extent of public health misinformation we have seen circulating.”