Sleep summary data and totalMinutesAsleep not matching stages

[Fleet]

I'm pulling sleep data with the web api and the summary data for a record's sleep levels does not match up with the actual items within the levels array.  I searched and found this this thread and the "answer" was that the data returned was the "classic" type and not stages.  

I'm getting stage data with my return value and the levels and summary still do not match. 

Below is a recent data pull, with dates removed so the post goes through.  This is using the endpoint /1.2/user/-/sleep/date/:date.json, with the actual date replacing :date.

{
    "sleep": [
        {
            "dateOfSleep": "<removed_for_google>",
            "duration": 19680000,
            "efficiency": 99,
            "endTime": "<removed_for_google>",
            "infoCode": 0,
            "isMainSleep": true,
            "levels": {
                "data": [
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 180
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "deep",
                        "seconds": 1170
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 90
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "rem",
                        "seconds": 1170
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 3000
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "rem",
                        "seconds": 1020
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 1620
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "deep",
                        "seconds": 420
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 600
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 480
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 2700
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "rem",
                        "seconds": 1620
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 2670
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "deep",
                        "seconds": 870
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 630
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "rem",
                        "seconds": 330
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "light",
                        "seconds": 270
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 870
                    }
                ],
                "shortData": [
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 30
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 60
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 30
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 30
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 90
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 60
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 30
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 90
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 60
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 60
                    },
                    {
                        "dateTime": "<removed_for_google>",
                        "level": "wake",
                        "seconds": 30
                    }
                ],
                "summary": {
                    "deep": {
                        "count": 3,
                        "minutes": 40,
                        "thirtyDayAvgMinutes": 65
                    },
                    "light": {
                        "count": 16,
                        "minutes": 189,
                        "thirtyDayAvgMinutes": 212
                    },
                    "rem": {
                        "count": 5,
                        "minutes": 67,
                        "thirtyDayAvgMinutes": 73
                    },
                    "wake": {
                        "count": 13,
                        "minutes": 32,
                        "thirtyDayAvgMinutes": 36
                    }
                }
            },
            "logId": 44630889691,
            "logType": "auto_detected",
            "minutesAfterWakeup": 0,
            "minutesAsleep": 296,
            "minutesAwake": 32,
            "minutesToFallAsleep": 0,
            "startTime": "<removed_for_google>",
            "timeInBed": 328,
            "type": "stages"
        }
    ],
    "summary": {
        "stages": {
            "deep": 40,
            "light": 189,
            "rem": 67,
            "wake": 32
        },
        "totalMinutesAsleep": 296,
        "totalSleepRecords": 1,
        "totalTimeInBed": 328
    }
}

 
According to the summary data there should be 3 levels of deep sleep, 16 light, 5 rem, and 13 wake.  Counting the data and shortData, the deep and wake levels match up; however, there are only 4 rem levels and 9 light levels. 

Furthermore, when tallying up the minutes by level, the deep and wake intervals do more-or-less match up with their summary values, with a one-minute difference in the deep level given the time differences in level changes.  However, the light and rem stages are off by seven and three minutes respectively; also, they somehow have more time with less intervals than their summaries indicate - light sleep comes to 196 and rem comes to over 70.  When I calculate the overall total it comes to 307, instead of the 296 returned.

From looking at the app it looks like the summary data is being used for display; I'm pulling Fitbit data for a research project that relies on further calculations using the level data itself, so while I can use the summary data for some fields, I don't think I can continue using the level data for a complete data pull until I clear up what is going on. 

Is this expected behavior, and am I misunderstanding what the level and/or summary data is showing?  Or is this something that needs to be corrected on the server side?  Any guidance would be appreciated.

[GordonFitbit · Accepted solution]

Hi @Fleet 

In the documentation, we have a section called Interpreting the Sleep Stage and Short Data.  It might help explain why there is a discrepancy in the summed values.   

[Fleet]

Thanks for point that out @GordonFitbit, I had missed that.  I was merging the shortData in but not recalculating the surrounding levels.  I've redone my algorithm and it does look like it's matching up as expected now.