All of our access token suddenly became invalid

[mdaubs]

Starting around 5PM Eastern yesterday evening, all of our users' refresh tokens began returning "Invalid refresh token" responses when attempting access token exchange. Prior to yesterday evening, nearly every user access token was being refreshed regularly and successfully except for a handful of users, a hundreth of a percent per day, which we chalked up to users deauthorizing our app. The refresh token values we have recorded look to be the same format we've been seeing since migrating to OAuth 2, 64 lower case hex characters. We haven't made any changes here since migrating our users to OAuth 2 a few weeks ago. Is anyone having issues that may not be reported on the Fitbit status page? Is anyone from Fitbit able to provide additional assistance or information that may be specific to our application account? Thank you.

[mdaubs]

We've been digging into this and it seems that a large number of errors for a small number of users compounded filled up our job retry queues. Combined with our rate limit logic against the Fitbit API this caused what appeared to be an issue refreshing tokens for all users but actually just a handful. The job queue + rate limit itself is what caused the issue to appear to us originally as affecting all users. So I think our interaction with Fitbit is working fine.

[JeremiahFitbit]

If you still need assistance, please contact us privately with your client id, user ids, timestamps, and logs of the refresh token you sent that was said to be invalid.