07-17-2019 10:34 - edited 07-17-2019 10:53
07-17-2019 10:34 - edited 07-17-2019 10:53
Today at 10:16 AM CST, we received quite a few 502 and 504 errors from Fitbit's Cloudflare proxy. Since then, we've seen a massive increase in "invalid grant" errors when attempting to refresh tokens -- we used to see maybe 1 a day where we couldn't refresh a token, we now have about 10 to 30 an hour.
Can Fitbit assist?
07-17-2019 11:31
07-17-2019 11:31
I noticed the same problem refreshing tokens, tons of these errors. Apparently this issue just started this morning.
Invalid Response: {"errors":[{"errorType":"invalid_grant","message":"Refresh token invalid:XXX . Visit https://urldefense.proofpoint.com/v2/url?u=https-3A__dev.fitbit.com_docs_oauth2&d=DwICaQ&c=iORugZls2LlYyCAZRB3XLg&r=gRR_t3sVc2lRWOMR3vKC9y4kASNHhI2hhEY1EcsZo6Y&m=o54VqRauLbLUHPX1AI7T5rSdhuhN2rQ5gvZGk8gEJlk&s=8im7OCA0VUZDryJwsGwZJIKaT4Inf-9JxJoVJkUmIoo&e= for more information on the Fitbit Web API authorization process."}],"success":false}
However, tokens are not really invalid, what's going on here!
07-17-2019 14:33
07-17-2019 14:33
It is now 4:31 PM CST and the situation hasn't improved. In addition to a high rate of "invalid grant" errors being returned, I am now seeing error 500's periodically returned from Fitbit:
<html>
<head><title>500 Internal Server Error</title></head>
<body bgcolor="white">
<center><h1>500 Internal Server Error</h1></center>
<hr><center>nginx</center>
</body>
</html>
07-17-2019 14:52
07-17-2019 14:52
Thank you for reporting the error. @jeaglemc I see in our logs that if executing the call again, you can eventually refresh the token. Are you seeing a situation where you cannot eventually refresh the token?
@alexperto are you experiencing the same thing?
07-17-2019 15:36
07-17-2019 15:36
exactly, executing the refresh_token again works.
so then, is this an API issue?
Thank you for looking into this @Gordon-C
07-17-2019 15:37
07-17-2019 15:37
When we receive an "invalid grant" we try again a few seconds later (it's been this way in our interface ever since we had to move to the refresh token process). Currently, this 2nd attempt is often successful but not always.
Today the 2nd attempt has been unsuccessful for 11 users. Certainly a small number, but to put it in perspective:
07-17-2019 15:56
07-17-2019 15:56
More details here, I've received around 633 invalid_grant notifications so far, I have only tried to refresh 5 of these tokens (5/5 successfully). I don't want to run a bulk script yet until I have a confirmation of this being solved.
07-17-2019 16:30
07-17-2019 16:30
@alexperto Would you please private message me your client ID?
07-17-2019 16:47
07-17-2019 16:47
@jeaglemc Thank you for the description. I see that in our logs and we're investigating.
07-17-2019 18:27
07-17-2019 18:27
@alexperto Thank you for sending me your client ID. Your scenario is very similar to @jeaglemc . We're still investigating the problem. I hope you have more information for you both tomorrow.
07-19-2019 06:23
07-19-2019 06:23
Hey @alexperto, I haven't heard anything but it appears the invalid_grant situation our system was experiencing was resolved at 4:00 AM CST yesterday morning. If you're still experiencing it, you may want to let Fitbit know.
07-19-2019 10:45
07-19-2019 10:45
Thank you @jeaglemc, yeah apparently everything is back to normal (partially).
Unfortunately, I just noticed in my logs that 886 tokens became invalid permanently which is a big deal.
@Gordon-C is it something you can do help with these tokens? I don't want to ask our users to re-authenticate, it's a huge number. 😞
07-19-2019 11:47
07-19-2019 11:47
Never mind, I re-run the refresh-token script and only 2 tokens are invalid 🙂 so we are good
08-05-2019 16:03
08-05-2019 16:03
Hi @Gordon-C , apparently this issue occurred again today, Is it something you already are aware of?