@MrMarv
@iccir919
@eglons
@craigvan
There has been a lengthy discussion about the use of the JSON file format over on this thread, which is now closed. https://community.fitbit.com/t5/Fitbit-com-Dashboard/Resolved-Changes-to-Data-Export-tool/td-p/30687.... The tread has been marked as resolved but only because after a huge outcry the original CSV data export was reinstated. There will no doubt be more so here is another thread so we can learn more about the content, provide further feedback to the Fitbit team and hopefully get things fixed.
The JSON format is no doubt here to stay and if you want heart rate data it is the only way to get it. However, the current export file is not without its problems, here are some and there will no doubt be more.
- You have to download everything from day one each time you export. For some that is a huge file.
- Inconsistent data formats and types and standards
- Missing data
- No explanation of some of the data e.g “confidence” in the Heart Rate data
- Conversion from JSON to CSV format
Tools
The tools problem can be addressed 2 ways. If you have Excel in Windows there is a JSON to CSV add in. It only does one file at a time.
@ICCIR919 has produced an online tool that converts and merges data files of the same type. It also fixes the timezone problem making time local.
https://iccir919.github.io/fitbit-json-to-csv/index.html
It runs in the web browser on your PC so none of data goes anywhere.
Data Content
@mrmarv has also provided an overview of the data I’ve copied it here so you can see some of the issues for yourself.
Time zones:
GMT - Distance, Exercise, Heart Rate, Steps
Local Time - Altitude, Calories
No standard - and a company's reputation often lives or dies on its standards.
Start Dates of the monthly bucketed files - looks like a holdover from the 30 days in a Fitbit month. Maybe starts with the day of incorporation, I don't know. I could be missing something here. Could be a serialized date formula on the system they work on.
Files with a sequential number as part of the key:
The exercise logs have the oldest logs with a zero and the latest, in my case with 500.
The food logs have the newest with a zero and the oldest with, in my case 3300
Standards, people.
Calorie logs - no comment, other than it is in local time zone.
Distance logs:
GMT
No clue how they decide to book an entry - the time differential between the zero distance entries is sporadic
The value appears to be in centimeters - no biggie but interesting that other logs are not in centimeters.
Exercise logs:
GMT
My treadmill entries show distance in miles NOT centimeters
My device initiated walk shows distance in miles NOT centimeters
Duration is in milliseconds - really?
Food logs:
ZERO nutritional values for customized foods. And let's face it people, the only way to track nutritional values and get them right is to customize them. Fitbit is not even close to being perfect and neither is MyFitnessPal or FatSecret so don't try and point me there. Especially after the "Nutritional Label Fiasco" a while back.
Fitbit has already shown weakness in breaking down nutritional values for display on the Web and on their apps. This just shows their developer and project leader's blindness once again.
How are people going to get data to present to their doctors when Fitbit is no longer reporting on these very important values? Or even evaluate their own food plan? Can you say "Disservice"?
Heart Rate logs:
GMT
Confidence level of 1 - 3 - I'd like a definition.
