Ionic vs. Charge 2 heartrate and sleep inaccuracy

[Davidbit]

I just got myself an Ionic! 

However, from the moment I started wearing it I felt like the heart rate wasn't quite right. When checking my wrist, the Ionic was about 8 beats per minute ahead of what I counted myself. To be sure I wore both my old charge 2 (on my right wrist) and the Ionic (on my left wrist) for a day.

Here's a screenshot from me waking up that morning, Ionic in blue, Charge 2 in red. Every horizontal line is 5 beats per minute.

Now, what's going on here? How can these two trackers have such different results? I've been wearing them both according to the through fitbit recommended instructions. 

Sleep also isn't entirely similar (note that they aren't the same length so are a bit skewed at the end):

So, well, who's lying here? And how can I be sure that I've got some thrustworthy data to work with? I really looked forward going to the Ionic from the Charge 2 but now I'm doubting both. 

Any info will be appreciated,

Kind regards,

David

[bbarrera]

---

@Venkats wrote:

@bbarrera

Sleep stage accuracy has been independently verified by John Hopkins university and there is a published document. fitbit technology is 96% accurate with respect to a EEG based system.     

---

@Venkats I searched and can't find that anywhere, can you point me to it? There is a paper published summer 2017, but the accuracy was in fair to good range (based on my limited understanding of Cohen's kappa).

[Venkats]

@bbarrera

Here are two URLS.

https://pdfs.semanticscholar.org/a18d/14fbc724c2dfe982fa4b44fbae1d5948c68b.pdf

http://www.mobihealthnews.com/content/fitbit-slims-down-heart-rate-sensor-offer-alta-hr

Per the second one fitbit's design (sleep tech) is based on a panel of experts from Stanford and John Hopkins and as such I myself will discount the evaluation because it could have been done by the same panel members. I didn't read the second URL when I replied you.

[bbarrera]

---

@Venkats wrote:

@bbarrera

Here are two URLS.

https://pdfs.semanticscholar.org/a18d/14fbc724c2dfe982fa4b44fbae1d5948c68b.pdf

http://www.mobihealthnews.com/content/fitbit-slims-down-heart-rate-sensor-offer-alta-hr

Per the second one fitbit's design (sleep tech) is based on a panel of experts from Stanford and John Hopkins and as such I myself will discount the evaluation because it could have been done by the same panel members. I didn't read the second URL when I replied you.

---

@Venkats

The first link has the following data, from a study published in 2012:

"Fitbit and Actiwatch-64 showed 97.8% and 95.7% of agreement rates with PSG for sleep detection but 19.8% and 38.9% for wake detection, respectively [6]. Finally, both devices were found to consistently misidentify wake as sleep and thus overestimated both total sleep time and sleep efficiency."

There is nothing about accuracy of sleep states. Only that Fitbit Flex has 98% accurate at detected sleep (vs wake), and 20% accurate at detecting waking up.

The second link has this nugget:

"Fitbit plans to submit data on the use of its sensors to detect sleep stages at SLEEP 2017, a major professional meeting of sleep specialists, later this year."

which takes you to this conference abstract:

http://www.sleepmeeting.org/docs/default-source/attendee-documents/abstractbook2017.pdf?sfvrsn=2

page A26 has the Fitbit authored abstract (not independent research). It states 69% accuracy of identifying wake/light/deep/REM, and a Cohen's kappa of 0.52 +/- .14. There isn't agreement on interpreting those scores, but some put the lower end 0.38 (0.52-0.14) as being poor/fair (wikipedia) and upper end of 0.66 being "fair-to-good/substantial."

Nothing here supports your idea that Fitbit sleep stages is 96% accurate, in fact Fitbit's own research shows sleep stages has "hit or miss" accuracy.

[Venkats]

@Davidbit

Last night I wore old Blaze on right hand (marked as dominant in the profile) after registering it as a new user and ionic on left hand. I extracted data from 2130 to midnight of 11/1/2017 and produced the comparison below. From the raw values it does appear that ionic has a tendency to show a slightly higher heart rate, even at rest (lying down in bed).  

[bbarrera]

@Venkats thanks. Do you plan to swap wrists and repeat the experiment tonight?

[Venkats]

Did you mean wear both the watches on same wrist or swap the watches, wear blaze on left while having ionic on right. 

I will do what you want but the point is Blaze (on right) matches with my BP monitor while ionic (on left as I would wear the watch normally) seems off by 3 beats ofcourse that was not a continuous measurement.  

Once can arguably say 3 beats is too low and statistically insignificant but when seeing what happens during workout, it seems rather significant.

[bbarrera]

Swap wrists. Never use two optical HRMs on the same wrist and expect to get valid results. We all hope to see the same 3 beat difference when Blaze/Ionic are on opposite wrists. I vaguely recall someone doing a similar test earlier this year and not getting the same difference. So just curious, if you don't mind. Thanks in advance.

[tghand]

I read a review of an attractive $28 Chinese fitness watch with lots of features that described it as "A nice, pretend fitness watch."  That's how I feel about looking at the Fitbit products and their heart rate inaccuracies.  It really boggles my mind that they don't fix this issue in their products because they are getting hammered for it.  And it IS possible to have an optical wrist heart rate monitor that is reasonably accurate.   Check out the graphs here:  https://www.dcrainmaker.com/2017/06/polar-m430-in-depth-review.html    [click on link at left for hear rate accuracy]  I do like the full-featured pretend website that Fitbit has (which is largely based on faulty heart rate data) and kind of hate to leave the family, but polar, with its accuracy and chest strap option, is making me think twice.  I may wait and see if the Versa accuracy is better before I leave...

BTW, does anyone have any information on Versa heart rate accuracy from a review somewhere?