Losing steps - Surge and One

[davisfoulger]

I use both a Surge and a One. Before you conclude that what I'm about to report has anything to do with that, please note that these events occurred in locations where there was no possibility of the Surge and One overriding each other. I've lost steps before in what were clearly accidents of priority between the devices, and in both directions. That has made me pretty carefully how I sync the devices. I think that problem could be resolved, at least in part, pretty simply with a feature that allowed you to specify when you are switching devices (and in which direction), but that's NOT what happened today.

I lost steps in syncs of my Surge at least 4 times today. The first time it occurred was between 7pm and 9pm. I was wearing the Surge at the time. The One was over ten miles away and had been 10 miles away for several hours in which I had synced more than once, so it couldn't be a factor in the loss of steps. What is interesting, however, is that they weren't lost everywhere. The exercise log is my app and dashboard shows an 19/18 minute walk starting at 17:42pm (but with 0 steps and 35 calories, and zero active minutes. I did take a walk at that time. I walked briskly enough to sweat and for about 19 minutes. I took roughly 1800 steps during that walk. Somehow the synch process correctly credited me with the exercise interval, but lost the steps, active minutes, and calories burned in the interval. Again, the Surge and the sync device were ten miles away from the one and all the steps taken between 6pm (about 300 more) and 8pm were lost. This also impacted the count of hours with 250 or more steps, Both the 6pm and 7pm fail to reflect the steps that I took in those intervals..

The other three losses all occurred within a fifteen minute interval after 11:45pm tonight. The symptoms are pretty much the same. The One was over 30 feet away (well beyond the 10 foot range you recommend observing). The Surge was on my wrist and my sync device was in my hand, inches away. I had a pretty minimal goal for the walk. I was at 59 active minutes and wanted to reacy my daily goal of 60 active minutes. So I initially walked for about 11 minutes (long enough to get the active minutes recorded. That including climbing 8 sets of stairs (with 7 sets of stairs correctly recorded and synced). At the end of the interval the watch was showing over 8300 steps. After the syncing the app and watch showed 7700 steps. I knew I'd lost steps but still wanted the active minutes so I kept going, raising the step count to roughly 8100 steps. The next sync reduced my step count to 7500 steps. Again I kept going, raising the count to over 7900 steps when midnight arrived. After synch my step count was reduced again to about 7500 steps. So putting all the together, I took upwards of 1700 steps over the course of 20 minutes, but was credited with 63 steps, 36 calories, zero active minutes, but a a 20 minute walk that included 7 flights of stairs.

This isn't a dual device syncing problem. It's a serious intermittent data loss problem that can occur repeatedly within a 15 minute window. I think its a bug that needs to be looked at more seriously.

To underline that I'm going to present things another way. I do roughly 2200 steps to the mile. My final numbers for the day show almost five miles (or roughly 11000 steps), but I'm only showing 7500 steps. The roughly 3300 missing steps would bring my total up to roughly 10800 steps, which is about what I should have had going almost five miles. I'm also showing to roughly 20 minute walks with no or negligible numbers of steps, calories, and active minutes. That suggests that some elements of your back end handling of the stream are broken and others aren't. Milage estimates seem to work fine. Exercise logging seems to work fine. But steps, calories burned, and active minutes all seem to be capable of data loss even as other things are being correctly reported.

If I was a programmer (and I am) I would probably find a lot of useful clues in that.

[davisfoulger]

I think the problem is related to a combination of parallel processing and the accidental missequencing of data at the back end database. In particular, if an exercise event long enough to produce active minutes occurs and is saved to the database first, all the rest of the data is dropped by the parallel data streams.

For the data I lost yesterday, floors climbed, mileage, and active exercise intervals (data I did not lose) would have been saved to the database before steps, calories, 250 step hours, and active minutes (data I did lose). and the loss is a simple function of throwing data away because the processes observed that there was an active exercise interval recorded.

My thesis is that data is lost when active exercise intervals arrives at the database enough ahead of other data for that data to be rejected based on the existence of an active exercise interval. That would explain other losses that I've seen on other occasions. I've lost credit for 250 steps in an hour more than once. I've lost credit for stairs climbed at least once. Yesterday I lost steps, calories, 250 step hours, and active mnutes (which presumably were rejected based on the calculated active exercise interval), but did not lose stairs or mileage.

I would note that this sequencing issue could well have an impact on the processing of dual Fitbits, especially if they are synced in parallel.

[davisfoulger]

Another data point and a simple suggestion.

I have, at this point today, taken over 1700 steps. Most of those steps come from three talks, one each between 10-11, 11-12, and 1-2. It still should not be possible for the One and the Surge to be conflicting with each other. The One was at least 20 feet away for my most recent sync and does not appear to have synced. The sync device and dashboard are showing a reasonably correct statement of steps, but I'm only showing one 250 step hour when I should be showing three. Worse, the 15 minute interval step graph doesn't show any steps for 11-12 and almost only 11 steps out of over 300 for 10-11. Indeed, it appears to have lost some steps that it used to show at 3am. Even worse, the dashboard is showing fewer steps for 1-2 than the app is.

It's entirely possible that dualing devices can account for the loss of the overnight steps, but as I sit here my sync device can't see my one (never mind sync with it), so I really have no idea what the problem is this time. That was true when I did my most recent sync as well, but I see that the One was synced recently anyway (perhaps as I sped through the room it is in.

To the extent that it might be dualing devices I will make a simple suggestion. Add a button to turn syncing of a device off and on without removing the device. Removing is easy (I just removed my One), but adding it back is a pain that takes minutes to get through. I would suggest putting this button at the top at the top of the syncing devices page for each device (remove is at the bottom of each page). Adding that button and some associated last sync when a devices sync is turned off and an associated first sync when it is turned back on should make it a lot easier to avoid sync conflicts and debug the problem.

I just "removed" my One. I actually dread adding it again, as I will do sometime in the next 24 hours (for what I hope are obvious reasons related to the Surge's charge time. I'm doing that with the plan of repeating what I just did and seeing what happens. I plan to try something else that might help as well.

[davisfoulger]

Another data point. My roughly four hour experiment with the One removed from my sync device was surprising. I thought it might solve the problem. It did not.

The good news is that there were no problems while the One was turned off. I collected more steps in three hourly walks, two of which overlapped the hour by enough to give me 912 steps, 151 calories, 15 active minutes (bringing me up to 37active minutes), three 250 step hours, and a 16 minue exercise interval. I was, overall, up to about 3700 steps on the day (halfway to me daily goal).

Having synced the Surge I then turned on sych again for the One.

The sync resulted in the immediate deletion of about 1200 steps, bringing my total for the day down to about 2550 (a third of my goal for the day). The details of how it did this are as surprising as the deletion was. All of my steps before noon had been deleted. All of my incremental calories above baseline had been deleted. All of the miles from before 1pm had been deleted. The flights of stairs I did before 1 PM hwad been deleted as well. The 250 step intervals I'd done between 10 and noon disappeared as well, but all of the data after 1:00pm remained. Note that there is an inconsistency in this. I turned off sych to the One at about 2pm (after the 1:00 data was synced. Z

Clearly, data from six hours earlier should not have been deleted, especially given that the One had been synced for that period already.

I've already suggested that syncing of dual mode devices needs of off switch (and on switch). These results reinforce this notion. The user knows which device they are using. The sync device should pretty much know this as well. Either way, it ought to be pretty easy to protect already synced data from being overridden with zeros from a device that hasn't been in use. That's especially true if I've just added it to the sync device. All of suggests the following:

1. The first sync to a Fitbit that has just been added to a sync device should be write (tor the Fitbit) only. The same logic should be used if, when a Sync On/Off feature is added, when Sync is turned back on for an existing device.
2. When a a Fitbit device is removed from a Sync Device, any data on the Fitbit device should be wiped so that it shows 0 steps, calories, etc. There may be value in doing one last sync if another Fitbit device is not in use, but users have the opportunity to do a sync before they turn off sync from the Fitbit, so this isn't necessary. The wipe can be write only to the Fitbit that is being removed.

Bottom line. It appears that a lot of this problem can be solved (or at least put under user control) with an Sync On/Off button and associated logic, It's a logical next step in the sync software.

[Rich_Laue]

There is only one stream of data when it comes to stats, using the exersize tracking mode only sets an event type and records the start and stop time. If using the gps then a second stream with this data is sent.

[wmchapman]

I'm with @Rich_Laue, I believe the second GPS stream when using the Surge vs a single stream step count from the One may be where your conflicts of calculation and interpretation are coming from.

Shutting down the One makes no real world difference, on or off your person, since there is no Master.

Try back-to-back non-tandem days JF with exclusively One and then exclusively Surge (steps only, no GPS) and compare these norms to your former data - I'm curious as to the deviations norm to former.

Keep us in the loop and keep moving! Excellent observations.

[davisfoulger]

Status update:

First, thanks to those who indicated that the data moves from sync device to server as a single stream. That's encouraging, but may not be entirely relevant to my thesis. If the data goes into the database as a single write, it might not be. If the database write breaks the data up as it writes it in various tuples (database terminology), it could be. If the database writes are done in discrete classes (in effect, different streams) as would be done Multi-tasking object-oriented programming languages like Java and C#, it would be highly relevant. I don't know the details of what Fitbit does in its programming.

Second, something changed over the weekend. 0 - Before saying anything, I should note that one thing changed over the weekend. I stopped seeing steps deleted from my overall step count after the fact as I was seeing on Friday and Saturday. The overall step counts were reasonably accurate on Sunday and Monday, but the 15 minute step counts remained highly inaccurate. As a general example on Sunday I recorded over 13,000 steps, but only 7000 or so of those steps were reflected in the 15 minute graph. Most came from a single nearly one hour walk through Central Park. That had impacts on active minutes (low), 250 step hours (only 4 recorded out of at least 9). The same sort of thing happened on Monday, where I show an 18 minute exercise interval in which I supposedly took only 22 steps and burned only 36 calories, but I actually took several thousand steps. I think (but can't be sure) that the overall calorie count is reasonably correct. The stair count is correct. All this suggest that there is still a problem recording 15 minute interval steps that continues to have reparcussions for derived data (like 150 stop hours).

Third, Fitbit followed up with me on this and asked me verify that my Fitbits were counting steps correctly. Both are counting steps reasonably accurately. Each of the test intervals I did on the Surge were within three steps of my count (all low). Each of the test intervals I did on the One were correct within 2 steps (all high). More accurate would be better, of course, but in my view these measurements were close enough. I've never had a pedometer that was completely accurate. Accurate within 3% works for me. All of the data from these test intervals streamed to my sync device and Fitbit's servers correctly and, for the first time in at least five days my first two 250 step hours recorded correctly. It's a little early to assume that means that the streaming of 15 minute interval steps has been fixed. As I indicated in my reply, nothing in what I reported above suggests a problem with the count on the devices.

My initial concern was that my step count for the day was being adjusted downward by thousands of steps a day at the server and that several derivative numbers (15 minute hours and the integrity of computed exercise intervals) were being contaminated as a result. Since Sunday my primary concern has been recording of 15 minute interval step data and problems with numbers derived from that. I will continue to track my data closely while things are repaired.

As a side note, I can only presume that repair of the overall step count means that a programmer worked on the weekend. I know that, as a programmer, I never particularly enjoyed solving code problems over the weekend when I could be spending time with my family. So Kudos to the people who solved that much of the problems on short notice. I hope that Fitbit will properly acknowledge that special effort.

[Rich_Laue]

As a python programmer I'm aware that "tuples" unlike "lists" can not be changed. 

I'm not sure what fitbit does, but suspect that the data recorded by the tracker does not get modified but has an associate list that is referanced, with the modifying instructions. 

[davisfoulger · Accepted solution]

All of these problems appear to have gone away. Today at least my step counts are right, my 15 minute data appears to be correct, my miles, calories, stairs climbed, and active minutes appear to be correct, my active minutes appear to be reasonable, my 250 step hours appear to be correct, and my 15 minute or greater active exercise intervals appear to be right.

I'll report back if that proves to be wrong in the future, but my presumption is that the programmers at Fitbit successfully have fixed the problems I reported. I thank them for their hard work and hope my clues helped.