How accurate is HRV?

[SunsetRunner]

How accurate is the HRV in the health metrics?

Moderator edit: updated subject for clarity

[SallyOz]

@Compendium I have read a great deal on HRV and have an understanding of some of the metrics required to accurately measure it. Due to the readings I was receiving on Charge 5 (and on the other 3 Charge 5s that I bought for other people) I bought an ECG grade chest strap heart monitor and I have been recording my HRV from my Fitbit and my chest strap every day. 
The chest strap takes 12 different metrics direct from the heart. The Fitbit shines a light on your wrist. Which one would you trust?

The chest strap even shows the balance between your sympathetic (fight and flight) nervous system and your parasympathetic (feed and breed) nervous system. Both devices give a readiness score. The chest strap is used by elite athletes. The readiness score on Fitbit and the chest strap have never agreed. 
I think the Fitbit gives great sleep analysis and 24 hour heart beat. I use it for that. Its HRV score is suboptimal (and that’s being polite). 

[SallyOz]

@Shelly0872 what Compendium is saying is true. It’s just the HRV data from the Charge 5 doesn’t capture all the data necessary to give an accurate HRV. It will basically indicate stress and tiredness because it records your sleep and your activity and uses that data to calculate a number. 
I have been following some chat sites where serious athletes (I am not one! I’m just interested in heart health). And some of these athletes use 4 different devices and capture all the data on comparative spreadsheets (they are serious!). They discuss the different apps as well as the different devices (and there are many of both). The most trusted HRV device is the H10 chest strap. Maybe give it a try?  I use 2 devices every day and I’m writing a paper on it. 

[SallyOz]

 @Compendium and @Shelly0872  these are some the metrics, direct from the heart, required to calculate HRV.
Sorry Fitbit says the screenshot contains numbers that are forbidden so I can’t post the screen shot 😞

[SallyOz]

@Compendium and @Shelly0872 I had to change the name of the screenshot so it didn't contain numbers. These are 12 of the metrics used to calculate HRV. 

[t.parker]

@SallyOz mind that overnight HRV is different from what EliteHRV shows you (you comparing apples to oranges) due to vastly different time domain. The morning HRV in short time domain is a different result. On the other hand, EliteHRV isn't suited for overnight measurement.

The metrics above are not used to calculate HRV. What is used are R-R intervals (because this is all you need). Metrics you see in EliteHRV are result of running different computations on R-R intervals. One thing you don't see here (but you could export from H10) are those R-R intervals.

[SallyOz]

Thx @t.parker Yes, I am aware of the different time domain. I was illustrating the complexity of the metrics required to deliver an accurate HRV score. I use both. I, personally, trust data straight from the heart, rather than via the wrist, as do most blogs that I am reading coming from serious athletes (I am not one). Most have two devices. I only purchased a second device when my results from Fitbit had me near death (8 and 11). Glad I did. Just sharing what I learned - thought it might be of interest to this group. Others may consider themselves to be in poor health when, in fact, they are not. The H10 is compatible with other apps and I have tried a few with similar results. 

[JamoC]

The H10 is a good companion for the Charge 5. There are many times the Charge 5 is spot on with my H10, the problem being not all the time. 

The Charge 5 does not analyse the HR enough compared to other pulse oximeters like whoop or Wellue ring.

The Charge 5 also has a "runaway heart rate" problem. I'm waiting for the "we've cancelled AFIB detection" email. I bet the cardiologist are loving it.

[JamoC]

HRV is 55 to 60 on H10 using app and 28 to 32 on Charge 5. 

[SallyOz]

Thx @JamoC that is really interesting that your Charge 5 and H10 do actually sometime agree! I have never seen that - not even close.

The Charge 5 jumps from a lowest of 8 to a highest of 32 with the majority clustering between 14 and 22. The H10 has a low of 48 with a high of 60 and clusters between 54 and 56. So no correlation at all.

I am guessing some of my unusual Fitbit HRV data comes from the fact that I am a 'non dipper' at night. My night time HR does not drop below my RHR and I think it confuses the Fitbit algorithm. I may be a very poor example of the Fitbit capability. Its nice to talk to someone else who has two devices. I find having two devices is common in the Polar community.

[SallyOz]

H10 clusters around 54 to 56 and the Fitbit 14 to 22. Today fitbit was 24 and H10 was 60.

Have you used the VO2max reading on the H10? I get a high score on that (for my age).

Seems those three little words accompany everything I do these days 🙂

[t.parker]

@SallyOz HRV isn't complicated to compute. If you get R-R intervals out you can calculate RMSSD and SDNN in Excel. These are not some magic formulas but well-known statistical methods. What is complicated is to tell whether there are signal artefacts. Smoothing data too much may result in lower HRV while not smoothing at all will result in huge fluctuations. Both methods are susceptible to large changes. It means that with different datasets, you need to work differently. That's why HRV won't be the same on other systems. You need to process the data first. The only thing that really counts is R-R intervals as this is the only thing that "comes from your heart". Everything else is processing which may vary on different systems. On Fitbit, with the EDA app, you can check your HRV on the spot in 2 minutes domain. Have you tried that? You can literally compare readings side-by-side.

I did 3 tests of HRV using three different devices in the time domain. It's important to understand what you're looking at. Fitbit works using RMSSD. EliteHRV uses both methods (RMSSD, SDNN) but the result is a score. The path is readings->filtering->method->output. Filtering and method may be different through various systems. If I try to check my HRV using Garmin paired with H10 and EliteHRV paired to the same strap, measure at the same time and very same time domain I may get the same results but it's not certain. R-R intervals will be the same but filtering the input and applied method may differ. I just did a such test and Fitbit has me 76ms, while EliteHRV (H10) is 61ms and Garmin (H10) 53ms. All three tests are done in a 2min time domain (I match that to Fitbit's EDA app). Which one then is correct? Two use H10 chest straps yet the results are different and ironically, Fitbit's reading came highest:

If you have EDA compatible watch, you can perform the such test yourself. Lots of data and numbers are great but it means nothing without understanding those numbers (and low HRV doesn't mean you'd be dead, that shows a lack of understanding of what HRV is, which means only your heart is closer to a metronome but it may mean anything).

Now, I'm not sure what you mean by "VO2max reading on the H10". H10 doesn't read VO2Max because VO2Max can't be read. It can only be tested by pushing yourself to the limit when you reach a peak of oxygen usage and above that, you go into anaerobic metabolism (the suffix Max is there for a reason, you must go at a maximum level of effort). The role of HR in that is to find out the anaerobic threshold but without proper lab tests, it's just a guesstimate. Without a proper test, you can only estimate 'what would it be if you pushed yourself into all-out effort". This is what Fitbit does and other platforms when users don't do proper maximum effort tests. How have you tested your VO2Max (Cooper test?). As I said, numbers are great but with numbers comes also meaning that it's good to understand 🙂 I do use VO2Max in my training, moreover, I have two VO2Max, one for running and one for cycling. This is because endurance capabilities differ between different activities. In one activity your aerobic capacity may be higher than in the other one. Originally, VO2Max was tested with running (already mentioned Cooper test) and used in different endurance sports but the problem was that a trained runner moving onto a bike (and being an untrained cyclist) couldn't perform that well, the anaerobic threshold was set lower, hence aerobic capacity, too (tiring quicker for the same amount of effort). My cycling VO2Max is lower than running. I can hardly elevate my HR on the bike to the same levels as I can do when I run and this is because I hit my threshold a lot sooner on a bike (lactic acid in muscles, switching to anaerobic metabolism). "Seems those three little words accompany everything I do these days" - same here with one difference - I understand what stands behind those little words 🙂

[Shelly0872]

Thank you 😊
Fitbit just isn’t accurate

Sent from my iPhone

[SallyOz]

@t.parker wow. Thank you for taking the time to give me all of that detailed information - I really do appreciate it. The fact that you have multiple devices is very interesting and that you can get 3 different reports on the same day, from the same heart, on different devices says a lot (to me).

I had considered getting a Polar watch to pair with my H10 which can collect and analyse other heart metrics. I am not an athlete. I have never played sport, never been to a gym or gone on a diet. My interest is in heart health which, of course, involves balance in nutrition, hydration and activity. I do have the sprinter gene and the endurance gene but have never had the desire to use either. I am an analyst (more into mental gymnastics). 

Wearing your Polar H10 and on the Polar Beat app you can take what they call a 'fitness test'. Wear the strap, lay down for a few minutes and totally relax. Once relaxed, tap the start button and lay very still. A voice gets you started and at the end of the 5 minutes it verbally gives you a fitness score eg: "Your fitness level is elite compared to those with the same age and gender ... etc" and at the bottom of the screen it gives a number - is this case it said "31 VO2max". If you have not found this feature please give it a go - I'd be very interested to how your result stacks up against your other equipment and your Cooper test (I have yet to google what that is).

My quip about being nearly dead was very tongue in cheek. I do understand that lower variability does get closer to the metronome. I have good basic training in anatomy and physiology. 

What you are saying reminds me of testing your DNA to establish your ethnic origins. I have done three tests, with 3 different companies, and they could be talking about 3 different people. 

Lots of emerging science to watch, compare, discuss and take with a grain of salt and a good dose of common sense.

Yes I understand what stands behind those 3 little words too. I am many things, but not an idiot. 

[Shelly0872]

💪👍

Sent from my iPhone

[JamoC]

@SallyOz 

Apologies, I meant in terms of exercise and hr. 

The HRV for C5 and H10 have never been close. 

[SallyOz]

@JamoC

Thx for clearing that up. I thought you had some magical alignment that I was unable to replicate.

I hope to find time to study other ways to use my H10 - at the moment I just do daily HRV through eHRV and Fitness Test through Polar Beat. I still log all of my C5 data daily - interestingly their monthly summaries and mine don't align on some data points. Still has some useful patterns.

[SallyOz]

@t.parker I researched Cooper test. Sounds like hard work. 

The Cooper test, in which you must go all out for 12 minutes, and there’s also the Yasso 800s speed workout that is designed to predict your marathon finishing time. Other tests, traditionally performed in a lab, measure oxygen intake and carbon dioxide output to accurately determine a runner’s VO2max values – the most common indicator of cardiovascular endurance.

VO2max is determined largely by genetic inheritance but is also affected by training.

Some studies indicate the Fitness Test I recommended (via H10 and Polar Beat) is reasonably accurate. I would imagine doing the hard yards is the most accurate.

“The purpose of this study was to determine the efficacy of the Polar Fitness Test in comparison to the laboratory test. Eighteen college-age students completed the Polar Fitness Test along with a laboratory test for aerobic capacity. The laboratory test consisted of a maximal Bruce protocol treadmill test while the subject was connected to a metabolic cart. The study found that the Polar Fitness Test provides results that are not statistically different from the metabolic cart results (t = 1.681, p = 0.111). Additionally, the 2 tests were strongly correlated (r = 0.545, p = 0.019). This indicates that the Polar fitness test may be an appropriate means of aerobic capacity testing for those not needing the accuracy of expensive laboratory equipment.”

[t.parker]

@SallyOz I had done the lab test on several occasions. The equipment is expensive but I'm not going to buy it myself 🙂 and booking in for a fitness test isn't that expensive nowadays (plus it's a job perk). Basically, this is the only measurement that can be called "measurement". Other tests provide estimated VO2Max. There is no way to know how much oxygen one uses without involving proper equipment to measure that. So you put a mask on that feeds you with oxygen (or to be more precise a mix resembling air but with known oxygen saturation that usually is aimed at oxygen levels at sea level) and tell you to run. Start slow, then the pace goes up. A different test is being used for that measurement as the Cooper test is designed for outdoors. Changing the incline and pace of the treadmill forces you to go harder. Eventually, the oxygen intake will stop increasing, you'll gonna reach a maximum level and despite increasing the effort that will not go any higher (it may actually drop but that depends). It's just that. Simple yet accurate measurement of VO2Max. Anything else is only a rough estimate that depends on the base sample used to collect data so an individual may be tested against that data.

Did Polar estimate work for me? Not that much. It's still a reasonable estimate, especially when I use Running Index (it uses maximum effort) so there is a little bit less guessing. Last year, I was starting my 12-week running program to get better on hills and did lab tests before and after. What I found then was that real VO2Max was slightly lower than any other system was telling me (I compared Garmin, Polar, and Suunto (this one was a bit difficult to work with as initially, my watch didn't support VO2Max at all so I was lacking a reference from before the training plan) and Fitbit, and all of them were giving me cheerful higher VO2Max 🙂 ). But then, after training my second lab test showed quite opposite result and it came up higher. Lab test came up with 65.2ml/kg, while Garmin gave me 58ml/kg, Polar 41ml/kg and Fitbit... well, can't tell as it doesn't store history 😕 Now the question is whether should I really care. No, because it doesn't change much for me. In every sport, there are more important metrics than VO2Max or HRV. When I cycle, I care about FTP, cadence and LTHR, running CP(FTP, but Power, not Pace), LTHR, and cadence (as form keeper and injury prevention), and climbing... I don't think there are any metrics useful in climbing :D. VO2Max is somewhere there but in reality, it isn't a game changer. It's nice to know but the more athlete you are the less you get fixated on the numbers that don't matter that much in the actual performance. Even with VO2Max what actually matters is a relative increase or drop but not an actual number because this is very genetically dependent. You want to know if there is a relatively huge/low drop or increase every now ends then. I do such tests before and after training plans which helps me estimate the relative effectiveness of that training but it can't work on its own. Only in conjunction with other metrics. Otherwise, it's rather useless (really, I never felt like changes in VO2Max ever impacted my performance in any significant way yet cycling or running FTP did a lot).

I noticed a tendency of giving to regular people (untrained, often not fit) metrics which used to be used by athletes and that causes quite a lot of harm (they tend to get obsessed with a single number). VO2Max is a metric that can be estimated. However, whether it is accurate or not for an individual can't be known until they perform the lab test. So speaking of the accuracy of such an estimate is rather pointless. How do you know that the sample population your results are compared against has similar genetics? In fact, EliteHRV with HRV score (it's a proprietary score so can't be compared to other systems on a 1:1 basis - dirty way to make athletes dependent on the platform, there's little more to be done in order to convert it but that's a different topic). Most of the athletes ignore VO2Max coming from sources like watches or any other non-lab tests (with the exclusion of the Cooper test done on the running track but here it's important to be able to estimate how hard is all out usually by HR+RPE). 

Without testing for aerobic capacity (all-out effort) it's just more guessing added to the result and that reduces potential accuracy. VO2Max must be done above the anaerobic threshold and most non-athletes (or less fit people) won't be able to perform that which means they have to take what they're being given - Fitbit's ranged score or Polar's Fitness Test score. Don't fall into confirmation bias. Because data matches your expectations it doesn't mean it's correct. Data must be always verified against ground truth. Without ground truth, it's just a guesstimate. If you consider this or another score correct then you do it because you choose to do so not because you know so. And it's ok, sometimes it's all people need to make themselves happy 🙂

Going back to HRV, I expect Fitbit may estimate R-R intervals from HR bpm as there was no hardware upgrade and even older devices received HRV (I don't think the sensor is capable of such a high-resolution sampling to perfectly detect peaks). Such estimation is possible but may reduce further the accuracy of readings (because there is no ground-truth reading of the R-R interval but input is already an estimate).

It is possible to extract the night readings from the Fitbit platform and test them against either H10 or OH1/Verity (with H10 I can extract R-R intervals). Running the same HRV algorithm on both datasets within the same time domain could prove whether Fitbit's accuracy is enough or not. Otherwise, it's just anybody's game.

[Shelly0872]

👍

Sent from my iPhone

[Shelly0872]

Really wow

Sent from my iPhone