Clarification on time zones and date/time values

[schaefba]

I work with Open mHealth, a non-profit that aims to bring down the barriers to integrating health data across different health data sources. We have worked with the Fitbit API for some time and we are currently refactoring our open source API aggregator (https://github.com/openmhealth/shimmer) and revisiting the API. As we've been working through that, we've had a one major issue around time zones that we were hoping we could get some direction on:

Responses from the API have a date and time, but no associated time zone. We understand that the only time zone information is stored within the user's profile, however that time zone can change over time. In your documentation you state, "all data is aligned with the user's specified UTC offset." Could you explain what that means? Does that mean that when the data is created it is created using that offset? Or is it created using the local time on the device? And when data is retrieved is it adjusted to the user's current time zone?

One more statement with regard to times and timezones in your documentation is also a bit confusing, "All date and time related fields in the API requests and responses are rendered in the local time of the resource owner's timezone." Could you explain what is meant by this statement? This statement to me implies that the "time" property in responses from the API will change depending on the current location of the user. Is that correct? Depending on the location of the user, the same datapoint could have a time value of "09:00" or a time value of "12:00"? And specifically which local time are you referring in this statement - the user's physical local time? The local time synced to the device due to physical location? The user's local time as according to the time zone they set in their profile?

I have one example that might be helpful for addressing both of the concerns above: if a body weight measurement was taken at 9:00a on June 1st, with the user physically located in PST, would the time of the data point read "09:00:00" when retrieved from the API? What if the user's timezone in their profile were set to MST, but they were physically located in PST? And if two weeks later a user were to travel to new york, and their profile time zone set to EST, if we were to look at the data point that was taken two weeks ago (on June 1st), would it still have a time of "09:00:00" or would it adjust to "12:00:00" to account for the new time zone of the device/user?

In addition, for endpoints that create on-going streams of data, such as intraday "activities/steps," how would time zone changes be handled? If a user were to be in one timezone at 09:00 on a day, then travel 3 hours to another timezone where it was 09:00 again (because they lost a number of hours), how would steps taken during the 09:00 hour be accounted? Would both steps be accumulated into the 09:00 timepoint? Or would the second set of steps be accumulated into the 12:00 timepoint because three hours of time had passed during their travel?

This begs a more broad question that would be really helpful for us to understand - are there any plans to add time zone support to the Fitbit API? If not, are there any options you can recommend for getting a precise, unambiguous date and time for datapoints? Our work aims to translate data into an open, platform-agnostic data representation and to do that we need to be able to know the correct, unambiguous instant in time when it was created.

I apologize for the lengthiness of this post and hope that these questions are clear. Thanks so much.

[JeremiahFitbit]

There is no concept of timezones on Fitbit devices. Whatever time the person's device states is the time that activity is recorded at. A weight measurement recorded at 9:00 AM will always appear at 9:00 AM, even if the user's time zone setting changes after the measurement was logged. Steps taken in the 9:00 AM hour will always appear then, even if the user's time zone setting changes later. (But note that this means that it is possible to have gaps of data and to overwrite data whenever the person changes their time zone setting.)

You shouldn't try to assign a timezone to any particular data point because your only reference for time zone is the person's current time zone setting, which may be different than the time zone when data was recorded.

Time zone support may be added in the future, but we don't have anything to announce now.

[ejain]

---

@JeremiahFitbit wrote:

Time zone support may be added in the future [...]

---

Yes please 🙂

[JeremiahFitbit]

We're aware of the benefit and we want to support time zones properly. It's a difficult thing to re-architect technically and experientially.

[Kevin_H]

"Difficult" is putting it mildly. 🙂 Calculating with time kinda sucks. For the studies I'm writing for it's really only the minute-level data that needs an unambiguous timestamp, a Unix (UTC) timestamp would work for our research projects, I'm inferring one when I store them now. Summary data doesn't really require UTC, but it would be cool if the applicable timezone were included, even as a separate field.

Oh, and sleep data. For version 1 it would have been nice to have a UTC timestamp on the sleep minutes, but I'm assuming you're no longer enhancing that. For the new stages sleep representation, the "datetime" element would need a partner element for the UTC time.

Also, I would strongly prefer an actual timezone as opposed to a UTC offset, e.g. "America/Phoenix" or "America/New York", but if offset is all there is that can be made to work.

Finally, it would be perfect if you made a few enhancements to your API to permit intraday queries based on UTC start/end or start/duration parameters.

I know this is a huge ask, but on the plus side we've loved having the low-cost activity data from the Fitbit API, and we're looking forward to many more integrations in the years to come! Thanks!

EDIT 11/29/2017: The inference doesn't work very well. When retrieving intraday activity for a user whose current user profile gives a DST-observing time zone we get minutes with "time" values between 02:00:00 and 02:59:00. Re-reading Jeremiah's post suggests that the time is stored as "local", and persistently. My hope that Fitbit stored the data with UTC and converted on demand was in vain. 🙂

But, it's FREE! (Thanks Jeremiah and Fitbit team!)

[jWhBOT]

Having data from the device with a timestamp field where time is always recorded in UTC and another field with a UTC offset or a time zone to specify which time zone a user was in would greatly improve the usefulness of this data. Without it, the data is unreliable and can provide false readings. I can’t imagine storing and presenting data in this was would be difficult. 

This will be critical when presenting data to people who frequently cross time zones and pull data at infrequent intervals. 

[Skatduder]

I travel 9 months out of the year. I'm happy with my settings.
--
Sent from myMail for Android

[pythonel]

6 years later, and the problem still has not been fixed.

I'm reading heart rate data at 1-second resolution from the API. Requesting data for, say, '2022-01-24' will return an array of (hh:mm:ss, heart_rate) tuples where the hh:mm:ss is "local time".

Subsequently changing my profile's timezone from Australia/Sydney (currently UTC+11:00) to UTC and re-retrieving today's heart rate data through the API: the (hh:mm:ss, heart_rate) tuples returned are the same, which shows that the data stored in the database is timezone naive. Furthermore, 11 hours of already-recorded heart rate data is missing, which means the API is using the profile's timezone to limit the end time of the data returned.

Changing the profile timezone back to Australia/Sydney reinstates the missing data.

It's unsatisfactory. Sydney has Daylight Saving Time, which when it goes into effect means the local time increments from 01:59:59 to 03:00:00, and when it finishes, the local time 02:00:00 to 02:59:59 occurs twice. It is this latter transition which will cause an hour of lost data - perhaps the first hour will be overwritten, or maybe the two sets of timestamps will overlap. Probably the bulk of the Fitbit user base will live in regions with Daylight Saving Time.

Heart rate data is collected potentially 24x7. A correct way to store the data is that each timestamp should be epoch time (seconds since 1st Jan 1970, 00:00:00 UTC). The API should then be able to return an unambiguous timestamp, and optionally apply a local time conversion.