Ensuring all data is retrieved

[BrentFit]

Excuse me this is question is very obvious, but I am brand new to using oauth and fitbit api.

I have succesfully got our app to retrieve data for the current day with a scheduler program using "fitbitClient.GetActivityLogsListAsync(DateTime.Now.Date); ". I have set this to run each day at 23:55

I have a few concerns

•  that since this only gets data for the current day, what if someone is in a different timezone and their activity is logged on the current day but they're in a timezone that means it's missed by the worker job. eg: someone is in timezone utc+ 3 . Lets say it's 01/01/2020. Our worker collects their data for that day at 23:55. Then that person does some activity and it gets synced as an activity on the current day(also 1 jan). On the next day (2nd Jan), the scheduler runs again but does not retrieve that activity on 1Jan, so we're missing data
• another concern is that our scheduler runs at 23:55, what if someone nlogs an activity at 23:59, it will miss that data.
• what if someone goes on holiday and doesn't sync, will the scheduler retrieve all the unsynced data even if it's from previous days?

Hopefully this is all handled by the api as it seems obvious, but I just would like to hear whether these concerns are valid or not. Thanks in advance.

[Gordon-C · Accepted solution]

Hi @BrentFit 

Unfortunately, most of this you will need to manage on your side.   We do have a rate limit of 150 API calls per user per hour, so you need to be mindful of that with batch jobs.   Anyway, there are a couple of ways you could accomplish this.

1. Within your batch job, you could query the current day and the previous day, then reconcile the data in your database.   This would allow you to capture the data during those periods when the user is in a different timezone or syncs after your batch job runs.
2. Also, I would use the Get Devices endpoint to determine when the user last synced their device.  You can track this information to know if the user skipped a few days and you need to query historical data.
3. You could use the time series endpoints, and specify the date and time when your batch job started or completed.  This can help with collecting data across timezones.
4. Lastly, instead of a batch job, you could implement the Subscription API to have your application be notified when a user syncs their device and they have new data to download.  Instead of a batch job running, your application would collect data throughout the day.  This will also help with managing the rate limit better.  Instead of executing a bunch of endpoints during a single hour, those endpoints are spread out and you're only collecting the new data that has been added.

Hope this helps!

Gordon

[BrentFit]

Thanks very much Gordon for your detailed reply, it helps a lot!