Sleep Logs Json response is missing level data for each minuteData entry

[linneverstops]

Hi Fitbit Community, 

I am trying to get sleep data using the Python API and I retrieved the data by calling:

agent.sleep(<base_date>)

 This returns a JSON object that gives me access to minuteData entries. However, each entry does not have a level associated with it:

  'minuteData': [
        {
          'dateTime': '21:51:00',
          'value': '1'
        },
        {
          'dateTime': '21:52:00',
          'value': '2'
        },
        {
          'dateTime': '21:53:00',
          'value': '2'
        },
        {
          'dateTime': '21:54:00',
          'value': '2'
        },
        {
          'dateTime': '21:55:00',
          'value': '1'
        },
.......

I get the total minutes of stages of the sleep in the summary tag:

'summary': {
    'stages': {
      'deep': 80,
      'light': 315,
      'rem': 119,
      'wake': 122
    },
    'totalMinutesAsleep': 600,
    'totalSleepRecords': 1,
    'totalTimeInBed': 636
  }

I do need the minuteData to indicate which stage it is in. May I ask why the json response I got is different from the API and how can I associate each minuteData entry with level or is there other API calls that I need to get sleep data from? Thank you so much!

Dennis

MS Student

[JohnFitbit]

Hi @linneverstops,

Welcome to the forums!

The response you're seeing with minute values is associated to a deprecated sleep endpoint we no longer support (Sleep Logs Version 1). I recommend that you change your code to implement the new sleep v1.2 endpoints.

The key difference is that the v1.2 endpoints support Sleep Stages, this is why you're not seeing what stages the user is in. Additionally, v1.2 displays the Sleep Stages in grouping rather than minute by minute details. To see how the new data is interpreted, please see here: https://dev.fitbit.com/build/reference/web-api/sleep/#interpreting-the-sleep-stage-and-short-data

I hope this helps. Please let me know if you have any additional questions.

[linneverstops]

Hi @JohnFitbit 

Thank you for the quick and helpful reply! I have two follow-up questions:

1) What do the 'values' in the minuteData correspond to? From what I can guess, a '1' indicates asleep and a '2' indicates awake?

2) You said V1.2 supports sleeping stages but in grouping only. Is there anyway for me to get the clustered-style data that was shown on the Sleep Logs API page:

Thank you for your help!

Dennis

[JohnFitbit]

@linneverstops can you provide me with an example of where you are seeing minuteData values? are we still referring to the v1 sleep endpoints or are you seeing this in the v1.2 endpoints? 

Unfortunately, there isnt a way to see the sleep stages in minute level detail. All sleep stage data will be returned in groups.

[linneverstops]

@JohnFitbit Right now I am pulling data using:

agent.sleep(<base_date>)

 I assume this is already the v1.2? If not, what method call should I use to retrieve data from v1.2 endpoints? I have failed when I used the time_series methods to retrieve data. It just said no such property as "sleep".

The data that I see are as below:

{
  'sleep': [
    {
      'awakeCount': 1,
      'awakeDuration': 3,
      'awakeningsCount': 17,
      'dateOfSleep': '2020-04-21',
      'duration': 38160000,
      'efficiency': 94,
      'endTime': '2020-04-21T08:27:30.000',
      'isMainSleep': True,
      'logId': <logID>,
      'minuteData': [
        {
          'dateTime': '21:51:00',
          'value': '1'
        },
        {
          'dateTime': '21:52:00',
          'value': '2'
        },
        {
          'dateTime': '21:53:00',
          'value': '2'
        },
................ <a lot of similar data lines>
        {
          'dateTime': '08:22:00',
          'value': '1'
        },
        {
          'dateTime': '08:23:00',
          'value': '1'
        },
        {
          'dateTime': '08:24:00',
          'value': '1'
        },
        {
          'dateTime': '08:25:00',
          'value': '1'
        },
        {
          'dateTime': '08:26:00',
          'value': '1'
        }
      ],
      'minutesAfterWakeup': 0,
      'minutesAsleep': 600,
      'minutesAwake': 36,
      'minutesToFallAsleep': 0,
      'restlessCount': 16,
      'restlessDuration': 33,
      'startTime': '2020-04-20T21:51:00.000',
      'timeInBed': 636
    }
  ],
  'summary': {
    'stages': {
      'deep': 80,
      'light': 315,
      'rem': 119,
      'wake': 122
    },
    'totalMinutesAsleep': 600,
    'totalSleepRecords': 1,
    'totalTimeInBed': 636
  }
}

Thank you for your time, John!

Dennis

[JohnFitbit]

@linneverstops 

agent.sleep(<base_date>)

This isn't a valid HTTP request and I'm not sure where you're pulling this from.

The example JSON response you provided is from the deprecated v1 Get Sleep Logs by Date endpoint.

GET https://api.fitbit.com/1/user/-/sleep/date/[date].json

The values you see for minuteData (1, 2, 3) represent: 1 ("asleep"), 2 ("restless"), or 3 ("awake").

In order to use the latest version of the sleep endpoints, you'll need to change the "1" in the URL to "1.2", like so:

GET https://api.fitbit.com/1.2/user/[user-id]/sleep/date/[date].json

The response for this endpoint should look like this:

{
    "sleep": [
        {
            "dateOfSleep": "2017-04-02",
            "duration": <value in milliseconds>,
            "efficiency": <value>,
            "isMainSleep": true,
            "levels": {
                "summary": {
                    "deep": {
                        "count": <value>,
                        "minutes": <value>,
                        "thirtyDayAvgMinutes": <value>
                    },
                    "light": {
                        "count": <value>,
                        "minutes": <value>,
                        "thirtyDayAvgMinutes": <value>
                    },
                    "rem": {
                        "count": <value>,
                        "minutes": <value>,
                        "thirtyDayAvgMinutes": <value>
                    },
                    "wake": {
                        "count": <value>,
                        "minutes": <value>,
                        "thirtyDayAvgMinutes": <value>
                    }
                },
                "data": [
                    {
                        "datetime": "2017-04-01T23:58:30.000",
                        "level": "wake",
                        "seconds": <value>
                    },
                    {
                        "datetime": "2017-04-02T00:16:30.000",
                        "level": "rem",
                        "seconds": <value>
                    },
                    <...>
                ],
                "shortData": [
                    {
                        "datetime": "2017-04-02T05:58:30.000",
                        "level": "wake",
                        "seconds": <value>
                    },
                    <...>
                ]
            },
            "logId": <value>,
            "minutesAfterWakeup": <value>,
            "minutesAsleep": <value>,
            "minutesAwake": <value>,
            "minutesToFallAsleep": <value>, // this is generally 0 for autosleep created sleep logs
            "startTime": "2017-04-01T23:58:30.000",
            "timeInBed": <value in minutes>,
            "type": "stages"
        },
        {
            "dateOfSleep": "2017-04-02",
            "duration": <value in milliseconds>,
            "efficiency": <value>,
            "isMainSleep": false,
            "levels": {
                "data": [
                    {
                        "dateTime": "2017-04-02T12:06:00.000",
                        "level": "asleep",
                        "seconds": <value>
                    },
                    {
                        "dateTime": "2017-04-02T12:13:00.000",
                        "level": "restless",
                        "seconds": <value>
                    },
                    {
                        "dateTime": "2017-04-02T12:14:00.000",
                        "level": "awake",
                        "seconds": <value>
                    },
                    <...>
                ],
                "summary": {
                    "asleep": {
                        "count": 0, // this field should not be used for "asleep" summary info
                        "minutes": <value>
                    },
                    "awake": {
                        "count": <value>,
                        "minutes": <value>
                    },
                    "restless": {
                        "count": <value>,
                        "minutes": <value>
                    }
                }
            },
            "logId": <value>,
            "minutesAfterWakeup": <value>,
            "minutesAsleep": <value>,
            "minutesAwake": <value>,
            "minutesToFallAsleep": <value>, // this is generally 0 for autosleep created sleep logs
            "startTime": "2017-04-02T12:06:00.000",
            "timeInBed": <value in minutes>,
            "type": "classic"
        }
    ],
    "summary": {
        "totalMinutesAsleep": <value>,
        "totalSleepRecords": 2,
        "totalTimeInBed": <value in minutes>
    }
}

The time-series endpoints are not available in the v1.2 endpoints as sleep stages are represented in groups instead of minute level details.

To see a list of the v1.2 endpoints, please visit: https://dev.fitbit.com/build/reference/web-api/sleep/

I hope this helps. Let me know if you have any additional questions.

.

[linneverstops]

@JohnFitbit 

I think I see where the problem is. I am currently using the Python Fitbit API from:

https://python-fitbit.readthedocs.io/en/latest/

Their API provided functions to autogenerate the HTTP requests to access your data. 

agent.sleep(<base_date>)

The above is the function to retrieve sleep data. I guess the library has not been adapted to the v1.2 endpoints of yours yet but instead has been requesting from the v1 endpoints without means to change the actual HTTP request. Guess I will have to create actual HTTP requests then. Thank you @JohnFitbit . 

Dennis 

[JohnFitbit]

@linneverstops Ah that makes sense. Since these are community-developed examples, it's up to the owners of each example to update them or the community requests for updates on the examples.

If you need help with implementing the endpoints, please let me know and I'll be happy to assist you further.

[linneverstops · Accepted solution]

So for those who came here for a solution:
apparently there is a way around it from their Github Issue Page. From "frejonb"'s suggestion, modifying the authenticated client this way will allow the calls to direct to the v1.2 endpoints:

authd_client = fitbit.Fitbit(CLIENT_ID, CLIENT_SECRET, oauth2=True, access_token=ACCESS_TOKEN, refresh_token=REFRESH_TOKEN, expires_at=3600000, refresh_cb=<callback function>)
# Updating the endpoint version to 1.2
authd_client.API_VERSION = 1.2

Referenced from: https://github.com/orcasgit/python-fitbit/issues/128

Hope this helps.

Dennis