Is Fitbit Sense for runners? - test of GPS

[t.parker]

I'm not looking for the answer to the question as I already know it, although it's not that simple. I decided to compile a small analysis from one of my recovery 5k runs.

Due to the UK lockdown, I stayed local and just ran a few laps to match finally 5km. I used:

• Garmin Fenix 6 Pro
  • Standard settings, no GLONASS, no GALILEO enabled, only GPS
  • Paired with Stryd (for the test, the real GPS distance is computed using a map)
• Fitbit Sense
  • Standard settings, no GLONASS (I think), only GPS
• MapMyFitness app on Android
  • Using a mobile phone GPS
• Stryd
  • Map from Garmin Fenix 6 Pro
  • Run parameters measured by the sensor

Word of explanation about Stryd and Garmin. Basically, the run parameters I could see on the watch during my run were obtained from Stryd (pace, distance, speed, power, etc.), however, for results, I use actual GPS data captured by Garmin and treat Stryd as a separate device of its own.

Results are quite interesting.

First 'the perfect' route designed in Strava, 4x1.18km lap + a bit:

The perfect route

GPS

First, I have to mention that the sky was cloudy however I ran in the open area. I will share all the routes:

• Garmin Fenix 6 Pro

Garmin Fenix 6 Pro - visible GPS drifting

This route is just wrong. It sort of "follows" the path but there is a lot of drifting as if the whole route was moved by some offset. In my book - it's a mess, horrible. I know that enabling Galileo helps a lot but to be fair, I didn't want to use it this time.

Measured GPS distance: 4.85km

• Fitbit Sense
  Goes off the track several times but isn't too badThis is the biggest surprise. It's in fact quite decent. Except for the path top-left near the pond where it all goes off the track and crossing the road (there's a tunnel, GPS was lost), Fitbit sense did a very good job. The GPS was acquired instantly, the loss of signal happened 4 times inside the underpass but the watch recovered quickly.

Measured GPS distance: 4.92km

Reported distance (visible in the Dashboard and app): 4.75km (I will comment on that)

• MapMyFitness on Android
  I think I need to change the phone...This is.... well... not just bad, it's just totally wrong in so many places. I'm sure I didn't use the power-saving mode. On top of that, MMF reported 4.6km.
  
  Measured GPS distance: 5.09km
  Reported by the app: 4.60km
• Stryd

In this case, there will be no map (the map used by Stryd comes from Garmin). I can only share run details:

Measured distance (from the sensor): 5.03km (bingo!)

The result is interesting as no device passed this test. Surprisingly, the closest to perfection was Fitbit Sense (who would expect, right?).

And that would probably answer the question if not a small detail that breaks this pretty picture of Fitbit Sense... There is a big difference between the GPS distance and what we can see in the app. It's in fact 170m on the 5km distance and that difference is going up. Why? This is because, like predecessors, Sense doesn't use GPS alone to compute the distance but some strange algorithm that includes steps and stride. It is easy to check - start stepping in place and observe the distance on the screen. You're not moving yet the distance increases. Also, the changes in stride affect distance computation greatly (which includes running hills). No matter what Fitbit says, a child can check that distance is not based solely on the GPS. It makes run parameters like pace rather useless.

It may be, the weather today had some impact on all the GPS devices I used, hence rather mediocre results. Nevertheless, the distance coming from GPS data only is very similar on all the devices. It's pretty close to 5km. Ironically, the best device that nailed distance measuring is a non-GPS one - Stryd.

If you are a serious runner then I would recommend Stryd + watch that connects to it (Garmin, Polar, Suunto - there are plenty). Garmin, on a good day or with Galileo on gives better results and reports distance/pace as they are, based on the GPS. If you are like me - not trusting the moody GPS technology, make sure the watch you go for will have the ability to connect with external sensors.

For casual running, when one cares mostly about burning some calories, Sense will do. No ability to use external sensors gives no chance to improve anything. Fitbit could easily fix the distance issue but for many years they are stuck in the world of steps which prevents Sense from becoming a serious tool for runners. Wrong data is useless data. Pity, because so far actual GPS hasn't failed me.

[Guy_]

Very good investigation and analysis, and interesting reading.

Thanks, it gets my vote.

[t.parker]

@Guy_ thanks. I should have also mention that even slightly bad GPS is still more useful when it comes to data than steps and stride. If you look at all the GPS tracks, despite lack of perfection here and there, the numbers fall not too far from 5km. For Sense it makes a big issue because if only GPS data was used, the difference between Sense and reality would be 110m (GPS 4.92km, reality: 5.03km). The thing is, this difference depends on the GPS, it may happen, may not and for different and longer route it may in fact be rectified on the go. While now, when Sense works with steps, the difference is 170m + 110m = 280m. It is 280m from reality. The longer run is, the more stride length varies, the 170m will increase. It's been something, I've been chasing Fitbit for long time since I noticed the distance computation was off on the Ionic. I would accept the GPS error, GPS can be moody, it's not the greatest tech and, as I shown above, other devices suffer similar GPS-related issues. **ahem**! In this test Sense beats Garmin Fenix 6 Pro when it comes to quality of recorded track! I wasn't expecting that happen at all! It is such a shame Fitbit makes recorded data deliberately worse. It's like shooting own foot and saying it doesn't hurt.

When it comes to running, Fitbit lacks lots of other things (for me, cadence is one of the most important), but plenty of runners look mostly at distance, pace, time - they keep it simple. They don't need device like Garmin with hundreds of parameters etc. If basics were right, Sense could actually satisfy such users. For me, lack of running cadence (I'm "cadence runner", it's my way of preventing my old knee injury to reoccur) and external sensors compatibility makes Sense a device not for me but there are many others who would be happy to have just this right: distance, pace, HR and time.

@Giampi71 I'm summoning you here 😄 What is your experience. As I recall, you have Sense and you're also battling Fitbit about the distance computation. Have you performed any tests since?

[marksurry]

Cracking analysis, thanks for sharing.

I find on my typical 5km run which is the same route every time, the Sense can vary about +-50m.  For what I require of the fitness tracking it fits my needs.

[Guy_]

@t.parker  I hope the points you raise reach the developers, it could improve things. They should hire you to beta test the improvements!

[N8teGee]

Great analysis, thanks for sharing. Just as a side note, I believe the Sense uses GPS+GLONASS as default (actually there is no way to change this).

Another way to confirm what you're saying is the case, is to start a GPS session in your car and drive for a few miles. You can see the map is plotted, but the distance stays close to zero.

Unfortunately this isn't the first time this has been pointed out to Fitbit (in great detail), but so far we have had little luck. Although it does now seem they are willing to confirm that GPS alone isn't used for distance. So I guess it's some progress. 

[Giampi71]

Thanks a lot @t.parker for calling me in this interesting conversation.

Back a couple of years ago I made several tests with Fitbit Ionic (at the time I owned a Tomtom Runner 3 that had very precise GPS sensor) and made a conclusion that the distance was based on steps by stride. I'm a very regular runner hence my stride doesn't change much (just to say Smashrun.com (the website I use to track my runs) in my last run awarded me a badge as regular runner with a pace variability of less than 5%). Therefore Fitbit algorithm wasn't too bad for me and the GPS on Ionic was quite good (not at par with TT although).

When last year I bought the Charge 4 I was utterly disappointed: terrible GPS sensor and a step based algorithm that made disasters at the point that I can't suggest the C4 at all.

So then I tried the Sense and found the same finding: step based algorithm to calculate the distance. I'm even more disappointed as the GPS is indeed excellent (at par with my Polar Vantage V2). However I gave the Sense to my wife (who is happy) as the HR sensor is a total disaster (however I saw you started another discussion on it so I won't bother).

What I'm really upset of is that I called so may times to explain the reason behind the algorithm and nobody bothered to provide any answer. Honestly speaking I feel nobody has an answer. Probably the coder of the algorithm had a good idea but eventually left Fitbit and now nobody bothers to investigate. Just my feeling.