Able to select which data to export and date range after Google migration

[NitLionPSU]

I recently migrated to the google app which has superseded my fitbit app. It appears that the google app does not provide the same options to export the data from the fitbit device that the fitbit app allowed. The original fitbit app allowed me to output a spreadsheet that was a daily summary of the body, activities and sleep information.

Moderator Edit: Clarified subject

[rockman112]

Updated the notebook so instead of 1 month's sleep data, it now converts all sleep data by concatenating multiple .json files into a single DataFrame.

It does this by pattern matching all .json files that start with "sleep-" and have the date format "{yyyy-mm-dd}.json" (suppose if your date locale is different, e.g. Europe, this might not work since you'd need to match {dd-mm-yyyy} or something, but for now, assume Google does it all the same).

Now, you should only need to provide Google Takeout's global data path for the 'takeout_path = ' variable and the notebook should do the rest.

There may be some 'NA' values for columns prior to 2020. I suspect Google or Fitbit made some sort of name change here, but any data after 2019 (i.e. Jan. 1, 2020) should be correct.

@Rajotter let me know if you do find those .json files. I believe from takeout.google.com I deselected all then only selected to export 'Fitbit'. Testing this now to double-check and confirm.

 

[rockman112]

Confirmed the Fitbit export via takeout.google.com contains the Global Export Data directory that also contains all the .json files, including those for sleep.

[AinATL]

Prior to switching to the Google login we were able to download a subset of our data or all of our data. Now all we have is the Google Takeout - which is less than impressive. If I had known I would lose functionality I would not have switched to the Google login.

[LizzyFitbit]

Hi @AinATL, thanks for taking the time to share this product feedback about restoring the ability to export your data for a specific time with us. I noticed this product feedback was already requested on this board, so I’ve moved your post here. Please support this product feedback by adding your vote, this helps our developers to keep tracking its popularity and demand over time.

[CJ-ISU]

I was super disappointed in Google Takeout.  I used this workaround which is a hassle but works well and allows me to export CSV files .

I switched my Fitbit to the google account and the data export functionality was terrible. I downloaded "SyncFit" from the Apple store to export Fitbit data to Apple Health (first I had to buy an old Apple 11 phone to get Apple Health since I have an Android).  I then used the “Health Export” Apple app to export csv files for my records.  This App allows specific data and date ranges.  It took some manipulation to correct the data. Sleep total start and stop time was correct but sleep events such as rem and awake were all off by 5 hours. After putting some formulas in Excel and then using pivot tables I was able to continue my multiple years tables of Fitbit data. This is the only solution I have found for Googles terrible data tool. Sad that Google cannot provide a decent export tool.  If this is the future of Google's artificial intelligence and other Google ventures Google will fade away.

[ShalFit]

Thank you very much for your willingness to help me but it is too
complicated. There should be a simple and friendly solution

[Henryhomp]

So is there a solution to being able to export say the last months steps, mileage and floors data? 

I was sent the link to this thread by FitBit support via Facebook Messenger.

It doesn't appear to offer a solution.

I completed an export yesterday, followed the link and downloaded the folder and files into a zip folder.

There's tons of useless information, but nowhere do I see my steps, mileage and floors data.

Why was the simple facility to export a months data, with fields selectable, into a csv or Excel file removed.

I really have had enough of the randomness of FitBit support, after being a user since 2016 it's definitely now time to a move to some other device.

[dargret]

@rockman112Thank you so much for this!!!

Any idea when you will update it to also aggregate the steps/distance jsons? I see in your comment that the plan is to go back and update it, but wasn't sure what kind of timeline you were envisioning.

[rockman112]

@dargret thanks for bringing this back to my attention 🙂 'steps-[date].json' has the count of steps for each minute, so those can be summed for whichever time aggregation you'd like (e.g. "Daily Steps").

'distance-[date].json' appears to follow a similar .json structure.

I don't think that should be a big lift, so I will set aside time for that in the next day or two. I will post back here when that is added to the notebook! 🙂

[rockman112]

All these files, anyone's FitBit data, can be found in [Your Takeout export's folder]/Takeout/Fitbit/Global Export Data/

All the .json files containing the data from the FitBit itself can be found down there, used to build an aggregation (like the old export method did!).

[CJ-ISU]

Any update on being able to export specific time period data as CSV files as was possible in the Fitbit app?  The Google takeout feature provides data that requires heavy manipulation to be of any use and has no date filter function. 

[dargret]

@rockman112 

Thanks so much!!! You have no idea how much time and agony you have saved me!

[rockman112]

@dargret I updated the [Jupyter Notebook](https://github.com/raymond-edgerly/fitbit_export/blob/main/fitbit_takeout_to_old_fitbit_export.ipynb) with 'steps' (month) and 'distance' (daily) aggregations. These can be changed by editing 'freq' for:

`df = df.groupby(pd.Grouper(key='dateTime', freq='ME')).agg('sum')

Change 'ME' to 'D' or whichever time offset works: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#offset-aliases

I think it's hard on the user to have them install Jupyter + run the notebook when all they want is a simple data aggregation.

I want to simplify this by making it easier to use. Something like hosting it online and having the user provide their Takeout folder and asking "What aggregation would you like for what domain (Sleep? Steps? RHR? etc.)" I'm not sure how long it'll be before I can make it that far.

To use the notebook on your local machine,

1. Download the .ipynb (Jupyter Notebook) itself from [Fitbit Export repo](https://github.com/raymond-edgerly/fitbit_export), or if you want to get fancy, `git clone` the entire repo to your local machine (explicitly, you only need to download the .ipynb to do your aggregations).
2. Install Anaconda to get Jupyter (Jupyter Notebook runs in Jupyter): https://www.anaconda.com/download/success.
3. Run Anaconda to open/start Jupyter.
4. In Jupyter, navigate to the .ipynb file and open it.
5. Once the notebook is open and you are looking at it, change 'takeout-20240330T183803Z-001' in `global_export_data_path = r'~/Downloads/takeout-20240330T183803Z-001/Takeout/Fitbit/Global Export Data'` to wherever your Fitbit Takeout export was saved.
6. Hit the double-arrow button near the top to run all of the cells in the notebook, and the rest should happen automatically.

You may receive a ModuleNotFoundError on first run for some cells. This is because Anaconda does not come with certain packages like PySpark or Altair. To install the packages, you'd want to use the environment.yml file from the repo.

Navigate in terminal (Linux) or command prompt (Windows) to where you have environment.yml saved and run:
`conda env create -f environment.yml` (will download and install packages specified in the .yml, e.g. PySpark, Altair)

Then run:
`conda activate fitbit_agg` (to switch into the environment)

Happy to help with any blockers or questions along the way 👍 Really, this should be Google/Fitbit's responsibility, but for now, it is one way to re-create the export and get your data in the format you need it in 👍

[rockman112]

As a side-note, my guess is Google turned aggregation export off because it was decided it would cost "too much" (whatever that number may be). It's cheaper to serve the customer their data and say "here you go, free to do what you please with it." Possible there was some other infrastructure/stack reason they decided to deactivate the old export feature, but my guess is that it was likely "computing resources" ($).

Not great to "switch it off," but maybe people become hesitant if they discover they can't get the aggregated version of their data.

The point is that, if they haven't restored the old export by now, they probably won't.

The bright side is that since you can access all the .json data, you can actually do more than you previously could. You can play with all the data. Certainly someone is bound to make some amazing health visualizations out of it. I was able to visualize my daily RHR average over a period of years. That code is also in the notebook.

[Roshak]

I would have never imagined that reporting would be worse and or absent following the move to Google. This seems to be a standard expectation of users and it needs to be added back as a feature.

[SFOTyke]

I wish I knew these export limitations before I migrated my Fitbit account to Google Fitbit. I got fed up of the badgering emails from Google to move my account, so I relented.

I used to download the previous month's data at the start of each month and cut-and-paste it into a spreadsheet, where I could track historical progress and create graphs. It looks like September 2024 was the last time I am going to do that, unless Google get their act together and provide the previous Fitbit export functionality.

This is my 4th different Fitbit tracker, but if this is not fixed when it is time for an upgrade, then I will look at other non-Google options.

[SlinStream]

Just entered Fit Bit world, i thought the problem would be accuracy of HR measurements during exercise compared to chest straps, turns out to be mostly OK. unfortunately the Fit Bit is useless when used together with other programs as data for specific exercises can not be exported.

This is rather unfortunate as it appear to be more of a policy than a software dificulty

[AllanSyd]

Wow.  I did not think that the functionality could get much worse after the transfer of my Fitbit account to Google.  How wrong was I!  The export options that were available are gone and all I can do is export all of my data.  What a regressive step this is.  Please do something to fix this ….. and everything that has been systematically destroyed since July last year.

[AllanSyd]

Isn’t it amazing that we have to vote for something we thought would be available when we transferred over to a google account from day 1.  If I had realised that this was the case I would have stayed with a Fitbit login until the bitter end.  I thought Google was a was all about innovation in tech but their implementation of changes with Fitbit and the apps etc makes me question this and remind me that they are not great at customer service either.

[bijan6]

I have been tracking data on at least 5 or 6 different Fitbit devices for nearly 15 years. I periodically will do a data export, mostly to have a backup, but my most recent data export request took nearly 3 days before I got an email saying it was complete. That is ridiculous - even for 15 years of data. It used to take maybe 20 to 25 minutes to complete when I could export the data in my Fitbit account. Can this be fixed as well?