Wearable devices are increasingly pervasive, and indeed the data generated by them is increasingly being used by employers and healthcare providers to monitor our activity levels.  The question is however, just how accurate are they?

I’ve already covered studies that aim to detect attempts to fake the data generated by using AI to spot erroneous behavior, but a recent study from the University of British Columbia casts doubt on the reliability of the data even when we’re not trying to fake it.

It found that the pedometer built into the iPhone misses around 1,340 steps during a typical day when compared to a purpose-built accelerometer worn on the waist.

“In order to make accurate conclusions, we as researchers need to know that the data is actually representative of real behaviour,” the researchers say. “That has major impacts in terms of patient care, and in terms of developing new and better research in the field.”

Testing accuracy

The study divided participants into two groups, the first of whom worked under laboratory conditions, whilst the second lived in regular living conditions.  Those in the lab carried two iPhones, the first of which was their own device, and the second was one provided by the lab.  Two were provided to test whether different models produced different results.  Participants were requested to walk on a treadmill for 60 seconds at various speeds so that the devices could measure their steps, whilst the research team counted their steps manually.

The results make fascinating reading.  The personal iPhones of each participant underestimated the steps by 9.4% when walking at a slow speed, and 7.6% when walking at a more normal speed.  The readings got better the faster the participants walked, but even at their quickest they were still off by around 5%.

Things got worse when the iPhone was tested alongside the accelerometer, with the Apple device underestimating steps by 21.5% across the entire day, which worked out at over 1,300 steps per day.

It should be said that this isn’t solely down to the technology, with a number of participants revealing that they didn’t have the phone on them all the time, but it does nonetheless underline some of the weaknesses in the technology, especially as much of our daily activity is at the slower speeds that the devices struggle with.

Suffice to say, whilst the results suggest we should be cautious about using the data from wearable devices for things such as health insurance until they get more accurate, the authors still believe they have many benefits for personal usage.

“For people who are already tracking their steps, they can rest assured that if their phone says they’re getting the recommended 10,000 steps in a day, they are probably getting at least that many, and they are working toward better health,” they say. “From a public health point of view, it’s better that it underestimates than overestimates.”