Resolved: Changes to Data Export tool

[KateFitbit]

Fitbit Update 11/30/2018: This issue has been resolved and I'll be closing it from further responses. 

---

Fitbit Update 11/28/2018: The old data export tool is now back and available for use. You can now choose which way you would like to export your data going forward. If you would like to access the old tool, please go here and select "if you would like to use the old data export tool click here" under Export My Fitbit Data. 

If you have any issues, please let me know. Thanks for your patience and reports! 

---

Fitbit Update 11/27/2018: Happy Tuesday! Just a heads up that we hope to have an update soon on the Data Export tool. I'll continue to update the thread this week. 

---

Fitbit Update 11/20/2018: While I don't have an update today, I can assure you that your feedback is being heard. One note, if you contact our support team, they will not be able to supply additional details, but I will continue to update this thread as I know more. We do appreciate the feedback.

---

Fitbit Update 11/16/2018:  Hi Everyone, here's an update on the recent changes to the export tool:

With the recent improvements to our tool, you can now export your Fitbit data, starting from your first step with Fitbit.

You can also access new data for download, such as heart-rate data, exercise logs, and your activity in the Fitbit community. For more information, see How do I export my data?

We understand that some customers are having difficulty interpreting their data in the new download format. We’ll continue to add features and make improvements based on your feedback.

Thank you all for your input in the thread. Our team will continue to monitor this thread and I'll let you know when I have further updates. 

[WiganKebab]

Fitbit Update 11/16/2018:  Hi Everyone, here's an update on the recent changes to the export tool:

With the recent improvements to our tool, you can now export your Fitbit data, starting from your first step with Fitbit.

Don't know weather to laugh or cry at this comment!! I'm due to replace my device to a new one and at the minute it's highly unlikely to be a fit bit!!

[Colinm39]

@WiganKebab That's the Fitbit Support reply I received plus a bit more.. I interpreted the extra as a positive spin and not addressing our concerns.

Still no reply from my 2nd email.

---

@WiganKebab wrote:

Fitbit Update 11/16/2018:  Hi Everyone, here's an update on the recent changes to the export tool:

With the recent improvements to our tool, you can now export your Fitbit data, starting from your first step with Fitbit.

Don't know weather to laugh or cry at this comment!! I'm due to replace my device to a new one and at the minute it's highly unlikely to be a fit bit!!

---

[lnt23]

First off, I want to say this was a terrible decision on Fitbit's part to change this export tool without warning or explanation. It sucks. And I question whether Fitbit's product leadership really understands or thinks about or cares about their customers, which is a real shame. When I've had issues with my trackers, the best I've gotten from Customer Service is an email telling me how much they care about their customers, but not enough to actually solve my problem. I love the product, but FitBit doesn't get it at all. 

Anyway...for those with interest, I was able to write an R script to convert the json's to csv. I'm kind of a noob myself, so it's basic but this seemed to get the job done. I put all the json files for one data type (eg steps) into a folder which is the C:/Users/mypath... below. The script grabs all the jsons at once and combines them into one. 

library(jsonlite)
filenames <- list.files("C:/Users/mypath...", pattern="*.json", full.names=TRUE) # this should give you a character vector, with each file name represented by an entry
steps <- lapply(filenames, fromJSON) # a list in which each element is one of your original JSON files
stepss <- do.call(rbind, steps) #combine the lists
write.csv(stepss, file = "steps.csv")  #write to csv

[dd2018]

It’s been mentioned the best way to get this problem resolved is to post our complaints on Twitter. I am not at all familiar with Twitter but I did Message Fitbit on their Facebook page https://www.facebook.com/FitbitUKIreland/ . (You do need a Facebook account to do this.) I felt the responses were very similar in tone to that which you hear from politicians, so you can make of this what you wil!!

“My 1st message to Fitbit on Facebook:

Hi. You have a ton of customers out there complaining about the removal of the Excel download facility. May I suggest you bring this facility back ASAP or risk losing a lot of customers!

1st reply Reply from Fitbit:

Fitbit UK & Ireland

We appreciate your suggestion, Dd. ! In regards to this, we understand that some customers are having difficulty interpreting their data in the new download format. We’ll continue to add features and make improvements based on your feedback. In the meantime, you can access new data for download, such as heart-rate data, exercise logs, and your activity in the Fitbit community. For more information please visit this page: https://help.fitbit.com/articles/en_US/Help_article/1133. Let us know if you have any questions, we'll be happy to assist.

My 2nd response:

Thanks for your response. I'm afraid the only way you are going to satisfy your customers is to bring back the option of having our stats downloadable with Excel. Yesterday I requested a download using the new system and responded to the email as requested but haven't received my stats yet, 24 hrs later. I notice in the info that stats make take several days before download is available!! This is not acceptable and I think you need to act quickly on this or risk alienating the majority of your customers. I get the impression that Fitbit are NOT going to reinstate the Excel facility, is that correct?

2nd response from Fitbit:

We understand that some customers are having difficulty interpreting their data in the new download format. We’ll continue to add features and make improvements based on your feedback. Our team will continue monitoring all of your comments and feedback and we'll let everybody know when we have further updates.

We understand that some customers are having difficulty interpreting their data in the new download format. We’ll continue to add features and make improvements based on your feedback. Our team will continue monitoring all of your comments and feedback and we'll let everybody know when we have further updates.”

[SunsetRunner]

I would guess this is Fitbit's way of complying with GDPR where is member is a right to request all personal data from a company to be supplied in a portable format. But it does not specify what the portable data should look like. So if Fitbit receive a data access request, they can just point to the new

format and that's it, they are in the clear. But they should have added this as another feature labelled "Data access request" and left the original export in place where members have a practical easy to use way of importing their performance data. They are 2 different things completely...

[dd2018]

I understand the Power Query Add in Excel is only available in versions 2010 onwards, so no good if you have Excel 2007 or less!

Yes, it's appalling and so frustrating that they don't give a **ahem**!

"Fitbit Update 11/14/2018: Hi Everyone - I've merged a few posts together to keep this topic in one spot. Our team is currently reviewing the feedback received on the update and I will update this thread when I hear further. Thanks very much for taking the time to reach out. "

A week has now passed since your post and no progress whatsoever in resolving the CSV download removal. Can you give me detailed instructions on how I am supposed to read large JSON files, all the free conversion tools I can find have file size limitations so are useless.

Moderator Edition: Format

[nightowl1016]

If there are no free tools or one that Fitbit provides that allows us to read the data effectively, I wonder if this change meets the GDPR requirements, as some have suggested is the reason for this change.

[hamlink12]

I work with Excel and Power Query just about everyday for my role as a data analyst and I have struggled to even use this new format. I love using my data I get from the export, so please bring back the easy to use CSV download format ASAP

[jcapille]

This is not a usable method.  Fitbit customers should not have to know an ounce of Excel coding in order to access our information.

[dd2018]

Here's another update on my fitbit Facebook message communication :

My message sent:

“A week has now passed since this post and no progress whatsoever in resolving the CSV download removal. Can you give me detailed instructions on how I am supposed to read large JSON files, all the free conversion tools I can find have file size limitations so are useless.”

Fitbit reply:

“There's a step-by-step instructions from Microsoft using the Excel to connect JSON files. Please see: https://support.office.com/en-us/article/connect-to-a-json-file-f65207ab-d957-4bf0-bec3-a08bb53cd4c0”

My response:

“This option is only available for for those users who have Excell 2010 or later. I have Office 2007 so I am unable to use. On top of that, even experts who have this facily are posting that it's a nightmare to use. Here is an example just posted on the forum: "I work with Excel and Power Query just about everyday for my role as a data analyst and I have struggled to even use this new format. I love using my data I get from the export, so please bring back the easy to use CSV download format ASAP" So it looks like that is a non starter! Any other suggestions please.”

Fitbit response:

"Office 2007 will not work to export the data but our team is also looking into this to improve the data export process on our end. We appreciate your patience and letting us know about this issue. Please stay tuned to this update.

Chat conversation end"

So, for all you folks out there without Excel 2010 or later, there isn't a real alternative yet to accessing your data.  I'm not confident there ever will be!

[MrMarv]

@dd2018 -

Good post - and once again nothing in their response recognizing that Power Query does not support any version of Excel for the Mac.

Or that their is a lot of missing data.

[snthompson]

I don't know who came up with the idea to change the export function, but (s)he should be fired.  It was the one thing that put FITBIT above others fitness trackers I've seen.  I called support and they wouldn't tell me why they did it. Does anybody know?  Is it time to go with another brand or will it be changed back?

[ArtemT]

I tried with Excel 365 and it still do not work.

Fitbit!

You made absolutely terrible decision! JSON file is not a generally accepted file, especially not for average users. Your response is even worse, the same message about "upgrades", "user experience", and "gathering feedback" is being copy pasted over and over again. While no resolution. Perhaps you expect I get a degree in Computer Science in the meantime.

I tried utilizing Excel 365 to export JSON file to Excel, it did not work. It needs to first be converted utilizing some other software first. Do you reasonable expect users to manually convert each file one by one for each day?!

Just make Data Export user friendly, that is all we ask for!

Moderator Edition: Format

[BarbKM]

I have excel 2013 and 2016 and have been unable to make any progress with these files.  The step by step instructions from Microsoft are about as useful as Fitbit telling us this is the new and improved version.  Is the intent to make us feel so stupid we stop asking for a reasonable file structure?

[PeterWombat]

Your post is a joke, right? You completely removed the ability to download data in custom date ranges (or recent fixed periods) into easily managed csv or xls files and instead give us piles of json? Not to mention you move it from an easy synchronous process to a bizarre sequence of email requests back and forth and at some point in the indeterminate future if you're feeling sufficiently generous I might hopefully possibly get some data? In json?

And you ask "could you share some details?" ???

Are you SERIOUS? 

Are you going to recommend we pull up the Python json library and code away to decipher our data? That might be an option for some of us, although even spending the time might be hard. I'll note I personally pulled me data and used the csv to feed into a PostgreSQL database that had my full history built up. I used that to run analytics, etc.  

So, ok, I could use Python to pull the json but hey, I already did a fair amount of work.

I guess WE no longer own OUR data.

[KateFitbit]

Your feedback is heard and Fitbit is actively reviewing this thread, as I have said several times. I will update as hear further. 

[PeterWombat]

Thanks MrMarv. I'm glad I found your post because I've just started to try and decipher the mass of json files I just received. I've been trying to find document that provides descriptions of the specific meanings, date/time usage, what units are used in the entries, etc.

I have not been able to find such a document. If you have found one that would be great.

Like you, I've spent most of my career writing and selling software. I understand the process of parsing json but can only do that if I understand the contents.

What I *want* is what the csv used to give me. I would get a summary PER DAY of my steps, floors, various minutes spent lightly/moderately/etc active, calories, kms (miles). That allowed me to easily add those entries to my personal database. That allowed me to quickly compare my exercise over time, compare sleep, distance, etc. I don't have particular interest in breaking down my stats to smaller increments, e.g., by the hour, etc. My schedule is relatively consistent so I know the times of days I'm walking, sitting, etc.

I have to assume that now that I have a pile of json files I *should be* able to parse those and build the same day-by-day summaries I had. But why does Fitbit want to force me to that?

[slakis]

I have the feeling Fitbit is just going to keep giving the runaround/generic "we're looking at future enhancements" thing and we will never get back what they took from us.

Nice, Fitbit. This is BS.

[JPSnow]

@lnt23 Thanks for sharing the code. Figuring that out was going to be my next step, so you saved me some time. Still a bummer that we have to this, even more so for anyone who doesn’t have the technical head start. And its still a pain to have to download all history just to get an incremental day or two.

[SunsetRunner]

---

@KateFitbit wrote:

Your feedback is heard and Fitbit is actively reviewing this thread, as I have said several times. I will update as hear further. 

---

@KateFitbitI appreciate you are only the messenger and probably in the dark as much as the rest of us but don't your management understand that if one of them was to stand up and give the community some sort of explanation as to why these changes were made and what was being considered going forward then the situation may be defused somewhat. They need to understand that your team being required to trot out the same BS replies isn’t helping.

Pete