Last year, demand for fitness trackers exploded as consumers rushed to stores and online sites to purchase the devices. But while many of them may be great for counting steps and monitoring your heart rate, a new study shows that they aren’t all that accurate when it comes to tracking the number of calories you burn.
Researchers from Stanford University analyzed data from seven of the most popular commercial devices and found that they varied widely in how accurate they were at assessing the number of calories users burned. However, even the best device was off by an average of 27%.
“The heart rate measurements performed far better than we expected, but the energy expenditure measures were way off the mark. The magnitude of just how bad they were surprised me,” said senior author Euan Ashley.
Inaccurate measurements
The study analyzed data from 60 participants who wore a fitness tracker while using a treadmill or stationary bicycle. The measurements on each of the fitness trackers – which included the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn, and Samsung Gear S2 – were then compared to measurements obtained from FDA-approved medical devices.
The results showed that all devices were pretty accurate when it came to measuring heart rate, with the average error coming in at or under 5.1%. However, the measurements for calories burned, or energy expenditure (EE), were far off the mark. The most accurate device, the Apple Watch, had an average error of 27% for this metric, while the least accurate device, the Samsung Gear S2, was off by an average of 93%.
While the researchers were unable to determine the exact reason for the devices’ inaccuracies, they theorize that it likely has something to do with trying to fit one algorithm to a wide range of people.
"My take on this is that it's very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone's fitness level, height and weight, etc.,” said study author Anna Scherbina. She points out, however, that heart rate data was probably more accurate because it relies on direct measurements.
Upload your data
To pursue the issue further, the researchers have created a website where consumers can upload their fitness tracker data so that it can be analyzed for a larger study. They say that the next step will be to see how these devices perform out in the real world instead of just in a laboratory setting.
The full study has been published in the Journal of Personalized Medicine.