03-11-2025 06:54
03-11-2025 06:54
Hi Fitbit Team,
We noticed that our Fitbit subscriber was disabled yesterday, 2025.03.10 at 19:01 UTC. Upon inspection, I see that the requests were successful right up to that point.
Can you give any information on what happened?
I restarted it today at 13:30 UTC.
Also, we don't receive a notification when this happens. Is that something that we should have received, or can configure?
Thanks,
Jeff Humphry
Answered! Go to the Best Answer.
03-12-2025 06:44
03-12-2025 06:44
I was able to find the problem in our logs. On 2025-03-10 from 18:15-18:30 UTC, we received roughly 600 "socket closed" exceptions. The means we tried to deliver the notifications but couldn't. We are currently investigating the reason why some people are not receiving an email notification when their subscriber get disabled.
Also, I noticed in our logs that we are consistency receiving a small number of "read timeout" errors from your subscriber. Are you trying to read and process the notifications as the come in, or do you put the notifications in a queue and process them later?
Gordon
03-12-2025 06:44
03-12-2025 06:44
I was able to find the problem in our logs. On 2025-03-10 from 18:15-18:30 UTC, we received roughly 600 "socket closed" exceptions. The means we tried to deliver the notifications but couldn't. We are currently investigating the reason why some people are not receiving an email notification when their subscriber get disabled.
Also, I noticed in our logs that we are consistency receiving a small number of "read timeout" errors from your subscriber. Are you trying to read and process the notifications as the come in, or do you put the notifications in a queue and process them later?
Gordon
03-12-2025 08:47
03-12-2025 08:47
Thanks @GordonFitbit!
We did indeed have an outage on 3/10, but I wasn't able to find those errors in the details, just a gap in successful requests (pictured above), so wasn't sure that was the issue. Is there some other way we can view history to verify?
The only operations we process on the subscriber endpoint are authenticate, log, serialize, and queue if authenticated successfully.
Best,
Jeff
03-12-2025 08:54
03-12-2025 08:54
Perhaps because the outage ended at 18:30 UTC, but the subscriber was disabled at 19:01 UTC, the errors dropped off the bottom of the details list. Looks like there is a max of 1000 displayed there.
03-13-2025 07:12
03-13-2025 07:12
My comment about the "read timeout" errors is an ongoing issue. They were occuring before your outage and afterwards. I just want to make certain your application is not missing any of the notifications that we are sending you. If you're trying to read and process the messages as they come in, then you're taking too long to do so. I would recommend you look into another option for processing the notifications you receive. For example, I would recommend spliting the read & process into 2 separate events.
1. when the notifications arrive, put them in a queue and reply with the 204.
2. as your system's resources permit, process the notifications from the queue and get the data from Fitbit.
We mention this concept in the documentation at https://dev.fitbit.com/build/reference/web-api/developer-guide/using-subscriptions/#Responding-to-a-....