Heart rate still inaccurate

[t.parker]

Since moderators encourage users to post new threads about the issue then this is what I'm gonna do.

As usual, another analysis. Exercise - indoor spinning. Progression training (each stage gets heart into the next zone and keeps it there for a while). I use 5 zones (one Anaerobic, I don't break it down into A1, A2 and A3). Fitbit zones are too simplified to be used for training.

Orange - Fitbit Sense, Red - PolarOH1 (optical), Blue - PolarH10 (strap)

Just a short explanation of the showcased exercise and chart:

Each horizontal black line is an HR zone separator. Each vertical blue line separates training stages.

Threshold and Anaerobic zones are very narrow, hence zigzagged line (I elevate my HR and let it go down by either stopping pedalling or just lowering gear - in other words, I can pretty much control my heart response, feels almost like a video game ;)). Anaerobic section drops a lot more because it requires more rest.

.

Used devices:

• PolarH10 (chest strap, most accurate) paired with Garmin watch
• PolarOH1- optical sensor, unsupervised (recording offline, extracting the data after exercise)
• Fitbit Sense - no explanation needed

PolarOH1 and PolarH10 are so close that both lines are pretty much overlapped. PolarOH1 has a few small hiccups but it could be used with no problem for HR-based training.

Fitbit Sense again disappoints. 

There is a huge difference during warmup and the first part of low-aerobic sections. I'm not sure why because it looks like Sense was going to kick start with other devices and suddenly it just dropped to <100 and started elevating from there. It took over 4 minutes to get back to more accurate readings. It's ~10% of the time of the whole activity.

The low-Aerobic stage, when Sense finally gained some accuracy is still quite messed up but let's say, acceptable. Nothing to be used for HR training but if somebody focuses on calories, that should be good enough.

Tempo/Endurance seems to be also more or less ok although there is one unexplained dip in HR. At least the device recovered quickly.

The Threshold seems to be problematic. The 3/4 of Threshold are probably alright on average. The HR is noticeably lower than H10 and the zigzag pattern doesn't exist. There are some dips, too. In the last quarter, however, all hell breaks lose! The Sense's readings are rubbish and this pretty much stays like that through the Anaerobic stage. It's the last 10 minutes of the activity (and higher intensity). This makes for 27% of the total time which is rubbish.

As for cardio and spinning (where cyclist doesn't experience any bumps on road, doesn't need to turn etc. so there is no extra noise), the result disappoints.

Link to Analysis #2 

[AtrainT13]

I am shocked that you were even able to get the sense to elevate past 130 BPM. I've been hardly able to breathe during and exercise and it says 120, which I definitely know is wrong using normal calculations of talk test and taking my radial pulse on the exact same arm as the sense. I wish they would let us know some status or what they are working on for the past year.

At this point, I am less than 14 days from purchase and I am going to give it another 25 prior to getting a full refund if we don't get an answer. Then I'm out of Fitbit for good. I chose to update from my versa 1 which did great, but I hated the design of the watch band connection. Fitbit is losing credibility in the fitness realm on many different forums from people who are serious about their fitness. It's unfortunate to see.

[t.parker]

@AtrainT13 this usually isn't a problem, my HR elevates well over 130BPM for cardio activities, however, for activities like bouldering, there is a lot more error. See one of my previous analyses (the second chart):

https://community.fitbit.com/t5/Sense/Heart-rate-inaccurate-on-Fitbit-Sense/m-p/5023675/highlight/tr...

Edit: I fixed the link

[AtrainT13]

It seems like anything that requires contraction of the forearms it's terribly inaccurate with. HIIT training with weights it doesn't know what it's trying to read. I manually took my pulse at 168 and the sense was reading 105.... I don't know why they chose to change the technology from the original versa, but they really seemed to have messed up this round as many with versa 3 are having the same poor experience.

[TommyIzzo]

@AtrainT13 They thought they had better, more accurate technology. I don't know how accurate the older technology is, and it can be easily manipulated, but it's at least motivating. Thing is, how did this release get thumbs up for distribution? Maybe Fitbit needs to look at their pre-release testing procedures?

[t.parker]

@AtrainT13 this is usually the case with all the watches. Bending arm or wrist and certain exercises (for example dead-hang) may cause inaccuracies and this is fine. That's why I have a chest strap to be able to track more problematic cases but steady-state cardio should be at least tracked with decent accuracy. In the example above, indoor cycling, there is not much bending wrist because there's no need to make turns or break or do really anything with hands (except flicking the gear shift from time to time). I know there are issues with weight training, bodyweight training etc. but as much the inaccuracy can be explained quite well for those activities, the steady-state cardio should work so much better. Yet, it doesn't (and sometimes surprisingly it does.... mind-blowing!). Frankly, I don't expect a fix anymore. If there was one on the way, it would be applied a long time ago.

[AtrainT13]

@t.parker that's the disappointing thing is that you aren't expecting one and they've been giving the same "We are aware and working on it answer for over a year now." It either seems unfixable at this point, or they aren't actually working on it and still selling it to unsuspecting victims at this point. I wish they would fix it.

[t.parker]

@AtrainT13 I don't expect any fix and I'd rather have the ability to connect external HR sensors. The wrist HR has limitations and there are cases it will never be accurate. With an external sensor, you can actually take off your watch. You don't really need to wear it during weight training (unless you follow the in-watch program and watch counts your sets and reps). I don't think there's a single person who likes to wear a watch and use kettlebells at the same time 🙂 When my exercises involve lots of wrist bending I prefer to take my watch off. Bouldering - it's just a bad idea to wear anything when on the wall (potential damage to the watch and it may contribute to injury). Wrist HR will never be perfect (but can get quite close). There are too many variables starting from skin colour, temperature, external noise, watch placement, sweatiness. It's virtually impossible to make it work for everyone and the wrist is the worst place to wear an HR sensor. I wish Fitbit invested time and money and bring this feature onboard. If customers have an option it could look totally different for Fitbit (and customers). I know that not everyone is comfortable wearing a chest strap but look at PolarOH1. It's a tiny armband. Just put it on and done. And there are plenty of other options available in the market supporting BT/Ant+ connectivity. I use wrist HR only for daily tracking. When I exercise, I use external sensors for accuracy.

[AtrainT13]

I would love to just have everything in 1 place where I can track the appropriate calories out vs. calories in and such as you can with the fitbit app. I'd take close for the heart rate with the wrist, but every since I received my sense, I haven't hit 30 active minutes in any workouts that I was getting 45+ with my versa 1. The versa 1 was much much more accurate in comparison to heart rate monitors and such.

Each time anything regarding heart rate gets posted, it gets moved under another post or deleted and given the response "we are aware of it and working on it." I just want to know what the plan or solution is. I feel like as the end user consumer we are supposed to be given that as it is something we pay for.

[AtrainT13]

@t.parker I think the most frustrating issue that I have with it all is that without the heart rate tracking like it should and has on previous models, it affects the user experience for everything. I cannot get an appropriate readiness score because I can't appropriately track my active minutes. I can't take advantage of the food functionality because I can't appropriately track my calories burned. My sleep score and resting heart rate are skewed. Thus, every piece of their software is null in void due to poor functioning hardware. Why have a free premium membership if I can't take advantage of it?

[t.parker]

Here's another analysis. All I can do for now (I'm self-isolating) is indoor cycling but will go back to analysing other exercises as soon as I will be able to get out of my house 🙂

This time, the goal is to ride a virtual route of 30km and stay below the threshold (Tempo/Endurance training). There are some exceptions to the rule due to several steep uphills to ride and this increased my HR into the post-threshold area but in general 84% of the time, I stayed in target HR zones and 16% in Threshold (not going into anaerobic) which is good enough for tempo training. What does it mean for HR? There are a few HR and power peaks but overall it was pretty steady-state activity (easy to track for Sense, I presume so).

Orange - Fitbit Sense, Blue - Garmin+PolarH10

 I have done several eyeballed spot checks and it clearly shows that sometimes I was just lucky (see the picture, it looks fine while it is just a middle of a bigger mess). Most of the time, Sense isn't too bad, it is just quite alright. It is visible lag compared to PolarH10 but even PolarOH1 has very slight lag (and from all the sensors I've been using only PolarH10 is really "lagless"). However, when I hit the first uphill, this is where Sense starts having problems and finds it very hard to recover. 18 minutes of the activity is totally messed up. The HR climbs were actually quite slow and smooth. We are talking here about 1-minute segments for which HR goes up 8-16 beats up. I dropped gear to lowest on inclines reducing speed and effort so HR could increase more steadily hoping it will be easier for the watch to capture it (and kept my arms very stable). Yet, Sense didn't manage to capture that correctly. For some reason, those uphills were the most problematic for the watch to capture and recovery from that state wasn't quick enough so the whole segment is pretty much trashed.

For a regular user, those errors should not matter. Even with those glitches, the outputs are pretty much correct. Zone minutes are still correct (this is just pure luck but hey...). Somehow Sense recorded a higher max HR (183bpm) while I never got anywhere near that. The average HR is 153 and should be 156. Those 3 beats will not make a significant difference for calories burnt. However, if it was my only device to do training with and use HR as one of the training metrics, the result would be very disappointing. It's 18 minutes of training filled with rubbish data which isn't reliable to be used in real-time.

Interesting thing is that spot check (like on the picture) is really meaningless and it's even less worth it without a good reference device (like counting HR using the old-school fingers method). You can't tell whether the readings are ok or not. You may only tell that reading at this moment of time is/isn't correct.

[AtrainT13]

@t.parker The inaccuracy and lack of stability is the most disheartening thing. My versa 1 that I traded in for the sense was much much more reliable. I thought with 'newer' tech it would be better, but it seems we've been duped at this point and time due to the difference between advertised expectation and actual reality.

You're graphs and breakdowns are really great to see and I would hope that Fitbit customer service, and the developers are looking at this and looking at this as a reference and it lights a fire under them to fix it ASAP as I wouldn't want my name tied to such a substandard product that doesn't deliver the traditional Fitbit quality that many fitbit members are used to.

[t.parker]

@AtrainT13 actually, I'm going to revive my old Charge 2 and try to compare two Fitbits of different generations. I'm curious about results. Tomorrow, I should receive the charging cable. I hope the battery will wake up 😄 I'm wondering whether the older tech will come up as superior to the newer one.

[AtrainT13]

@t.parker Doing just manual analysis of taking pulse the old way the versa 1 (Same-ish body type) was far superior to the sense. This was definitely true to max level anaerobic work where my HR would get to the 180's consistently. I was a collegiate rower and still push myself to max on rowing often.

[t.parker]

@AtrainT13 there is one caveat of taking pulse manually - it takes time as you need to take a full sample. It depends on recovery HR but if I try to check my pulse manually before I'm done my HR dropped already so what you're getting is really average over several seconds. If you check your pulse for 15 seconds (and multiply by 4) then 15 seconds you just get a slice of 15 seconds period (which also smooths data). You can shorten that period and use a different multiplier but the shorter reading you take, the larger amount of data in your 'manual buffer' is just made up 🙂 Manual check is pretty much only to be used in rest.

The way the HR monitors work is quite simple (at least those I'm familiar with, Fitbit doesn't reveal their method but I believe they didn't reinvent the wheel). Basically, there is a buffer that accumulates data until the current timestamp. When the beat is detected, it comes with the timestamp and is being queued. The oldest beats are being dequeued and any view on the buffer at any given time will provide a number of beats per minute. This is how PolarH10 has no issues with lag (although it is more complicated because PolarH10 is pretty much an ECG monitor, does a lot more work). If there is no data processing involved before delivering HR then the lag should be minimal and the data should not get smoothed. The Fitbit lag probably is a result of processing and trying to fix data (maybe infamous PurePulse2.0?). The processing isn't uncommon as there is a lot of noise coming in and raw data may contain even more errors if unfiltered. Still, it can provide, let's call it a rolling minute HR.

Manually, you aggregate isolated data and, especially with very high heart rates, such a check isn't for sure accurate (hence, I pointed out spot checks). That's why in my analyses I compare like-for-like and avoid spot checks.

[deiG]

I'm just curious. If Sense is worn on back arm as PolarOH1, are the results comparable to PolarOH1's?

[t.parker]

@deiG the answer is no. I have tested that a long time ago. The Sense (and other Fitbit watches) are not designed to be worn anywhere else. The forearm or bicep requires a lot more light to get penetrated. Of course, it depends on the person (maybe somebody skinny and with lots of veins visible will get some results, but for me, it didn't work). Sense sensor has two LEDs while PolarOH1 has six which are brighter and emit light with a lot higher frequency (120Hz, Sense is only 24Hz). Basically, the Fitbit sensor is designed to be worn on the wrist.

[AtrainT13]

It truly is proof that they owe us all a refund for such an inferior product that they currently marked as “better”. I wish the @@Fitbit developers take a look at this forum. 

[TommyIzzo]

Got my Versa 2 from Costco yesterday. Doesn't even have zone minutes yet, but out of the box, heart rate responds as expected. 😁

[deiG]

Please refer to the official declaration here.