ECG on Apple watch: Only 10 percent accurate...

[Tomtomato]

"A new study finds that only about 10 percent of Apple Watch owners who went to a doctor after being alerted of an abnormal heart rate were diagnosed with a condition.

False positives saddle consumers with unnecessarily medical bills, researchers said, and can overtax healthcare systems."

See: https://www.dailymail.co.uk/sciencetech/article-8798815/Apple-Watch-heart-monitor-falsely-warning-pe...

If even Apple can't get it right despite a serious head start on Fitbit, is there any point having it on the Sense?

[PureEvil]

@Tomtomato,

Why assume that Apple is any better at anything?  They had a working phone for years and then they released the just-don't-hold-it-that-way iPhone 4...  I'm just saying.

[Tomtomato]

Just because they have a head start and significantly more users than Fitbit. Therefore, a lot more time and data to adjust their smartwatches.

I guess the point here is that ECG doesn't seem to be reliable/work well yet on smartwatches, even from the leading vendor.

And stress detection seems to be a work in progress at best, so what the point of spending $100 for a Sense compared to a Versa 3?

As for your "just-don't-hold-it-that-way iPhone 4" example, I think Fitbit is just doing the very same now with "let-me-tell-you-how-to-press-an-invisible-button". I'm just saying...

[PureEvil]

@Tomtomato,

I think you're overthinking the entire button thing.

I have a Sense, the button is similar to the Charge 4, which is to say a groove in perimeter of the device.  This is what it looks like:

So you definitely feel it and know where to press and when you do, it'll give you a perceptible vibration to indicate that you've pressed the button.

It took no more than a few seconds to understand how it works.  Honestly, I think we've spent more time discussing this than it'll take you to 'get it' when you try it.  I don't think there are a whole lot of people crying foul at the button on the Charge 4.

As for the ECG, I don't necessarily subscribe to the ideas that:

• Apple is either the leading vendor in this space (maybe chronologically, but hey... so was the Betamax).
• Just because Apple can't do it well, it can't be done.  For instance, if you think about it, there's no reason Blockbuster should've been disrupted by Netflix, but it was.

I mean Apple also made the Newton and Apple Maps nearly killed people in Australia.

I think people are willing to tolerate more from Apple than other vendors (and not necessarily due to quality).  I mean can an iPhone 4 even be called a phone if it consistently drops calls?  Can you legitimately call something an activity tracker if it doesn't hold a charge for even a full day?

[SunsetRunner]

And you could say that Garmin is the leader in fitness tracking and activity tech yet their sleep tracking is nowhere near as good as Fitbit's.  I've had people ask why I don't wear an Apple Watch and my reply is usually that my usage scenario doesn't suit an iWatch with only 18 hours of battery capacity.  I can go for a run for a few hours and the battery on my current watch drops by about 5% with constant GPS and other sensors working.  I then know that I can safely sleep with it and not have to charge for several days.  Can you do that with an Apple Watch? no you can't.  Just because they sell the most smartwatches does not mean they're the best.

[Tomtomato]

With so many people complaining about the new "button", it was clearly a mistake or needs further adjustments.

Yes, Apple is the market leading vendor of smartwatches, by far, and they introduced ECG before anybody else, so they have more users and more data. Yet, results/accuracy is very poor...

[SunsetRunner]

@Tomtomato  it is very right to make this speculation. Apple has accumulated much experience to make its product as accurate as possible and yet they have to confront with difficulties. Fitbit falls into the same category as a newer competitor to apple.

@PureEvil you're also right that people tend to forgive the mistakes apple does. They created a status, a symbol, and people don't give up on this easily. 

Fitbit is like the interviewee; cannot afford to make even slight mistakes. And apple is the interviewer; got the job and plays the king, affording to do as many mistakes as possible without risking his position in the company. 

[PureEvil]

@Tomtomato,

First, on the button design, to use another example in your space...

Remember when the iPhones moved from the physical home button to the combined home button and fingerprint sensor?  There were initially many people who cried foul...  Did the users make the adjustments?  Yes.

Remember when the iPhone got rid of the home button entirely (was it on the iPhone 10)?  There were initially many people who complained and didn't think they'd be able to make the adjustments...  And did they?  Yes.

Do you even hear about the lack of a home button on the iPhone anymore?

The button design on the Sense is effectively the same as it is on the Charge 4, here's a picture of that:

I don't see a whole lot of people who are disproportionately distressed on that part of the Fitbit forum due to the change in button design from previous Charge models.

Second, on the ECG, just because there's someone who is already established in a space and even has more data doesn't necessarily mean that they're the 'heir apparent' and if they can't do it, no one else can.

Remember Friendster?  MySpace?  They had both the advantage of time and more information (due to having a head start) yet they were disrupted by Facebook.

Blockbuster could've easily branched their rental business to mail-in (the way Netflix started), they had the advantage of context information (which titles were most popular and to what degree), and yet they have been disrupted and have effectively shut down.  In fact, it is precisely the mindset that "we're the leaders in the industry... no one can touch us." mentality that led them to interpret Netflix as no more than a nuisance to their business... until they started to see their rental rates drop, by then it was too late.

This is pretty much the way you're describing Apple in this space.

If you want to talk about mistakes in feature design, consider this, as technology improves so do these spaces specifically:

• Battery capacity.  You can store a larger charge in the same volume or the same charge in smaller volume.
• Size and power consumption of the components improve.  You can accomplish more in less physical space that consumes less power.
• Introduction or improvement of certain sensors means that you can enable your device to do more.

Apple (and many other hardware vendors) make the very legit assumption that your users are willing to plug your device in everyday.  This is probably generally true...  You have to sleep sometime, you can't use your laptop or phone while you sleep, just plug it in then.  Similarly, you wouldn't use your Apple Watch when you're sleeping... Oh, wait.

Apple is using the very same "users are willing to plug your device in everyday" assumption to a device that doesn't follow the same usage pattern.  What if you get an AFib attack during that 19th hour when you're charging your device?

There are design decisions that Apple can make with the Apple Watch in order to get longer battery life...  Larger, heavier device to accommodate a larger battery...  Fewer pixels on the display that consume less battery...  A more modest processor that will consume less power...  Forgo a sensor on the device, since it will take up space and consume more battery... etc.

Many years ago (before the cameras on smartphones were good enough to take regular pictures), friends asked me what I thought was the most important feature of a digital camera...  My unwaveringly response was its physical size.  Simply put, if the camera is not small enough for you to carry consistently, it doesn't matter how many fancy features it has when it is sitting in a bag in your closet.

Similarly, can a fitness device be an effective health tracker if it's plugged in 25% of the time (6 hours of a 24 hour day)?  Are they collecting the appropriate data if 90% of their users do not wear it when they're sleeping?  By design the Apple Watch cannot physically collect data for a full day, doesn't this effectively nullify your 'more data' point?

[imjrode]

It's not meant to be an end all. It's meant to alert you if something might be up and get you to go to the doctor. Nothing is going to be better than a 12-lead ECG. I'm sure those in the 10 percent are extremely glad they had an Apple Watch to alert them of an issue.

[t.parker]

@imjrode this is all true. The watch, even with all the approvals needed in order to be considered as a medical device will not reach the same accuracy as 12-lead ECG. Apple Watch uses only two electrodes hence the accuracy won't be the same.

When detecting AFib, giving false positives isn't really such a bad thing as it triggers people to check themselves. The worst-case scenario would be getting false negatives which would leave people unaware of a potential condition. I know there was some research done on the chest straps, whether they could be used for the same reason (the contact area is under the heart and much larger, also no wires which reduce any noise) and the results were quite promising. Here's an example of such paper. The major difference between the strap and watch is that the first one can be monitoring for 24/7hrs (straps usually run on battery which lasts about 400hrs which is 16 days of 24/7 monitoring) and they provide accuracy also when we are moving (hence the use during exercise). I read a paper stating that some doctors consider using chest straps to monitor patients at home as this is the way of collecting lots of data with enough accuracy. Nowadays, some patients are being given a portable 3-lead device with some wires for such purpose. There is an interesting paper about its accuracy comparing with 12-lead EKG that can be read here. Even the 3-lead device excluded some patients with diagnosed Afib. The watch, no matter what, will not get even near such accuracy, hence 10% of real positives are still a good number. If Fitbit can give the same detection accuracy then it's still a good thing. The chest straps may replace one day the 3-lead monitors but the ECG in the wristwatch it's going to be just a toy for some longer time. Useful yet still a toy.

We are years before anything worn on the wrist will be able to obtain accurate data without a need for additional sensors. 

[N8teGee]

I agree with @imjrode. The Specificity is far more important than the sensitivity in this case, better to have false positives than false negatives.

Yes we have the issue of the so called "worried well", but once you understand the limitations of these devices I believe they can be quite useful. Even if you haven't got AFIB and the watch thinks you have, perhaps there could be other health/heart issues at play. I'm sure your Doctor will be able to decide if it's worth investigating further. The article mentions the following (below), so I think getting themselves checked over was the right course of action, even if only a small amount were diagnosed. 

"They found 264 patients said their Apple Watches alerted them to an abnormal heart rhythm. 

Half of them already had a history of heart trouble, and roughly two-thirds reported symptoms like chest pain, shortness of breath or dizziness.

But only 30 patients went home with a diagnosis of a cardiovascular event from their healthcare provider."

As for the button, I can't really see the issue myself (not that I'm saying people aren't having any). Perhaps it's because I've had both the C3 and C4 in the past with a similar button, and I'm now used to it. It's worth mentioning that the Sense/Versa 3 now have swipe gestures, so the button doesn't really have that much use. 

[SunsetRunner]

OK, people, I am retired medical and I can  tell you that expecting any device worn on your wrist that does not have multiple sensors stuck to the chest and is jiggling around a lot due to arm and wrist movement to give any really accurate measure of heart rhythm, is a fantasy. It will always be a best guess and not all that accurate. When medical people want to evaluate real life "at home" heart rhythm in patients where an issue is suspected they attach a device with multiple leads and patients wear that for a period of time. Example:

https://www.irhythmtech.com/professionals/why-zio

I get annoyed when watch features are called "ECG" or "EKG" since even devices like the Zio are not EKG, much less a silly watch.

They are good for screening, maybe, but as pointed out have large numbers of false positives so may wind up eventually with  "boy that cried wolf" problems. I would not rely on them for anything related to heart rhythm, to be honest. 

I would stick to things watches do at least fairly well like heart rate and (to a lesser degree) SPO2. Leave stuff like heart rhythm and BP to better systems.