Cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Google apps script for minute-by-minute data stopped working suddenly

I have been using a Google Apps script in a Google Spreadsheet to download minute by minute data. Today I tried to download data for my account and I got "We're sorry, a server error occurred. Please wait a bit and try again" at the top of the spreadsheet. Did something change in the API that might be causing this? It might be something with Google, have no idea how to debug it.


 I have another spreadsheet that doesn't do the intraday stuff that's working fine.

 

This script was based off of things I found online

Here it is https://jsfiddle.net/j72qbe08/

Best Answer
87 REPLIES 87

@CollinChaffin - have you made any updates to the module?  IIRC you had mentioned you might update to a GUI, etc?

Best Answer
0 Votes

@mattolomew I did forget to come back here to announce one big update - the automated ability to execute an entire month query in a single new command of ex: Get-HRmonth -Querymonth "2016-03"

 

I actually went one step further and wrote in the logic to only query and fill in the missing files - in other words, if you already manually for whatever reason queried 9 days in March, the above command first computes how many real calendar days are possible, then computes a list and subtracts the nine days already queried with existing Excel reports, and runs only the needed 21 daily queries.

 

I am working on other features.  The best way to track changes is to follow me on Twitter as I annouce them there, and/or Github where I do maintain a detailed changelog.  Also, if you use Slack, I even set up an automated Slack Github channel that sends immediate messages for any trouble issue opened, new release build, etc.  Shoot me a request if you want an invite for the Slack channel.

 

-Collin

Best Answer
0 Votes

HUGE life saver, Collin! This is a lot more robust than the Google docs method. Although, the default setup of plotting data in Excel is slowing the process down by a huge amount. I don't know powershell scripting, but after a bit of tinkering, I disabled the export to Excel (it should be straightforward to create a duplicate function that will just export to .csv) by commenting out the "Export-FitbitXLS" line in the function "Get-HRdata".

 

Additionally, I like the lightweight nature of gnuplot, which can very easily take in a .csv file and output the required plot in seconds. Matlab is also another powerful tool. But to make the plots even nicer, the order of the columns in the heart rate csv should be "Date, time, value" so that the Date + time columns can be combined in gnuplot (or in Matlab) as the x-axis. The current scripts create a csv with "time, value, Date" columns. I reordered the columns using this modified line of code:

 

#Reorder columns and write to csv
$output | Select-Object Date,time,value | Export-Csv -NoTypeInformation "$($Script:psFitb1tInvocationPath)FitbitHR_$($psFitb1tHRQueryDate).csv"
			


Thanks a lot for your efforts! It is totally unacceptable that Fitbit doesn't provide a more straightforward solution for their customers to download their data! They don't need to have to write programs and create "apps" on the dev website to get this basic info!!

Best Answer
0 Votes

Actually I have a ton of experience with Excel automation and unfortunately, the slow charting is because Excel is charting literally thousands of points for each data point - it has absolutely NOTHING to do with Powershell or the code, and I think I even mention that on the Github page.  In fact, the copy/paste method I use is up to a HUNDRED times faster than standard automation of placing each data value in each cell vs pasting and took me a lot of testing and trial and error to come up with for the module.

 

I would ask that since most users of the module may not be PS coders that you please do not post alterations of code here since that is really the purpose of posting it openly on Github - if you want a feature, the best way to request it is to open an issue on the Github issue list and perhaps I can add it.

 

The addition of only dumping CSV data is easy and since I already do export to CSV prior to the XLS, perhaps I will add a function to ONLY do the CSV data export and you are correct, all you really need to do is REM out the one single line that exports to XLS until I add a CSV-only function if all you want is the raw CSV data.   But bear in mind, most folks want the end product and until someone can demonstrate a faster, better method of charting the thousands of HR data points into a standard MS xls format - that is the reason my function currently performs both the CSV and XLS.  Actually, it was only for more advanced folks that I even left in the interim CSV export it performs - so I am glad you are finding a way to utilize the raw CSV data elsewhere!

 

The only new XLS method I would consider that I am aware of that is proven to be faster would be if I have more time and would be to attempt to raw write the XLS file based on the file standard and remove the Excel app from the equation all together - but honestly that will take significant effort.

 

If you can help put together a better way to get the CSV data into a XLS spreadsheet for folks faster than the current method I use, please post it on the Github issue and/or you could do a pull request/fork as well - I am certainly open to doing it faster but like I said, I spent a ton of time testing methods to come to the one my module is currently using.

 

Thanks for the input and I'm glad to see folks are using it!

 

Collin

Best Answer
0 Votes

Yes, I understand that the slowness of the Excel export is due to how excel does it and not something you have control over. I wanted to download the data for a whole month and with excel, it was taking a significantly longer period of time for each day. But without creating excel charts, I could download one month's worth of data in .csv format in about the time it created 1 excel plot. 

 

One idea to speed things up is probably to avoid using excel and instead use gnuplot. A disadvantage may be that gnuplot will output the plot as an image so you may not mouse over to inspect individual datapoints, but honestly with such a large Excel plot, it was very difficult (and slow) to inspect individual data points anyway. Gnuplot allows for command line scripting and can generate plots from extremely large datasets very quickly and as a bonus it is lightweight and opensource. 

 

I will try to see if I can create a script that will automate the process of generating plots using gnuplot. That could be a useful alternative.  Of course, the code snippet I posted is totally optional and is only to reorder the columns in the csv to make it easier to process in gnuplot. I am currently trying that out.

Best Answer
0 Votes

Hi!!!!

 

When I try to click on the Authorize, I receive this message:

 

Developer information: invalid_request - Invalid redirect_uri parameter value

Best Answer
0 Votes

I have found the problem,

the last version 20 of Library for OAuth 2.0 give this error: Developer information: invalid_request - Invalid redirect_uri parameter value

 

right now I am using version 19, and it works

Best Answer
0 Votes

Hi, Im new to Fitbit and these forums.  I purchased the Fitbit Charge HR in an effort to monitor my heart rate overnight on a minute by minute basis and was disappointed to discover that I could only see 5 minute intervals.  I was lead to these forums in an effort to find out how to get at this data.   Ive read through many posts and have to admit, I am thoroughly confused.  I have some software dev experience from many years ago but have never developed an app. With that in mind, I have the following questions...

 

Do I really need to create an app to do this?  I mean, its my own personal health data, why is it hidden from me?

Shouldn't an app already exist that allows me to get at this data?

Can someone walk me through it step by step?  

 

I would appreciate any help as this data would be of great help to my cardiologist.  Its very difficult to capture a short lived cardiac symptom on an event monitor that the docs provide since the event is usually over by the time the monitor gets connected. The fitbit data would help immensly.  Hoping someone can help. Thanks!!

 

Best Answer
0 Votes