SDK for Accelerometer Data in Esense2 for a project on Sleep at the University of Essex

[vitodefeo]

Dear Developers Community and Fitbit Official Support,

I am a researcher at the University of Essex. We recently secured £1.1M funds for a project about sleep and mental health.

For our project, we will recruit about 600 participants and they will wear a health band to keep track of their sleep quality and mental health issues. We are evaluation the possibility of buying 200 Fitbit Esense2.

Our main goal is to perform sleep stages classification. We know that from the Esense2 we can have some metrics to infer a person's activity level through the Web API such as activeMinutes, activityCalories, etc.  During sleep, we can have sleep stages (e.g. wake, light, rem, deep, restless, asleep) but we want to implement a more complex algorithm based on actigraphy (x,y,z accelerometer signals) and heart rate variability (HRV), recorded by means of photoplethysmography. Therefore we need to record also the x,y,z accelerometer signals.

Furthermore, we need to record the data continuously for 8 weeks and we will send the heath band to each participant via post.
The devices only store data for 5-7 days.  So, the participants of our study will need to sync their devices with the Fitbit mobile application on a regular basis so Fitbit can store the data.   Then we can pull the data at the end of the 8 week period using the Fitbit Web API.  We are aware that to pull the data through the Web API, we will need to have each participant consent to share their data with your application.

Our problem is that, unfortunately, the Fitbit Web API does not store the accelerometer x,y,z data.  That data is only available through the Device SDK which requires building a clockface for the Sense 2 device (https://dev.fitbit.com/build/reference/device-api/accelerometer/)
Is that correct?

All of the data you asked about, except for the accelerometer data, will come from the servers using the Web API.   The accelerometer data comes directly from the Fitbit device and you'll use the Device SDK to get that data.   

Given that we are going to buy a lot of Esense2 (about 200), we need to be super sure about all the SDK functionalities (e.g. accelerometer data from the clockface sync).
In particular, we need to understand how the clock face interface work.
Can we program the clock face interface in order to save the data to our server? If so, should we use an ftp protocol?
Can we, instead, save the data on Fitbit server and download the accelerometer data together with the other data (e.g. HRV) that we download through the Web API? I mean that we use the Web API to download all the data except the accelerometer data and something else (what?) to download the accelerometer data.
Can be the sync performed automatically? Should the user intervene, instead?

When you request the accelerometer data from the clockface, I think this is done at sync time.  The data is transferred to your server, but also transferred to our servers.   We use the accelerometer data to calculate the sleep stage information.  However, the raw data is not available through the Web API. 

Happy to have a Zoom / Teams  meeting with someone from the support.

Kind regards,
Vito

Moderator Edit: Personal info removed

[Gondwana]

This forum isn't monitored by Fitbit (Google) staff. However, the Web API forum is monitored by a Fitbit (Google) person. Given the scale of your proposal, he may be able to put you in contact with staff who could provide more official help.

To capture accelerometer data on Sense 2, you'd need to develop and install a custom clockface/app on the watches (so yes, you're right about the Device SDK). Fitbit hasn't released a SDK for Sense 2 that would allow this. Some developers have managed to get a beta SDK to work, or you could use an earlier Fitbit OS watch type.

Your clockface/app would need to run continually, since Fitbit OS doesn't support background processes. It would need to transfer data from the watch fairly frequently since accelerometer data is voluminous and watch storage is limited. Keeping this up for eight weeks is theoretically possible but, I fear, ambitious in practice. Basically, you have to write your own sync system for accelerometer data.

Web API syncing only requires user intervention if something goes wrong. Your accelerometer sync process can also run automatically so long as the user doesn't ever close the clockface/app on the watch, and so long as nothing goes wrong. It would be wise to expect problems and to put processes in place to deal with them.

Fitbit OS doesn't store accelerometer readings. It stores steps, etc, for up to 7 days, but discards raw accelerometer values as soon as they've served their purpose (eg, detecting steps). This is why saving and transferring such values would be up to you.

For the same reason, the Fitbit syncing process doesn't include accelerometer data. That's why it isn't available from the Web API or any other data export mechanism.

If you still want to use a Fitbit solution, there is some sample software you can look at here.

[Guy_]

Hi @vitodefeo - an interesting project but I definitely wouldn't choose a Sense 2 for it but rather a Sense.

There are two reasons.

Firstly there is documentation and an SDK for it to help with development and secondly it has a Gyroscope.

My SimpleSleep app monitors sleeping position during the night on a Sense using the Gyroscope which may be useful information for your study.

The objective of the app is avoid Apnea without needing special equipment (CPAP or other devices) and to monitor the effect on oxygen levels in different sleeping positions and to mainly improve the person's sleep and that of any partner.

A later version using the accelerometer is intended to actually detect Apnea and arouse the person to move. 

It may be possible in a later version to also use the accelerometer to detect position but it was already reliable and simpler with the Gyroscope.

[vitodefeo]

Thanks a lot, Peter!
I reposted this on the Web Api forum, as you suggested.
I understand that all this is feasible but I have to program my own sync system and ask each participant to keep the clockface always active. Furthermore, I would need an "alarm" system that warns me when a participant is not syncing. 
Thanks a lot for the link, very helful!
Many thanks,
Vito

[vitodefeo]

Thanks a lot!
I think we will go with Sense rather than Sense 2.
SimpleSleep seems very interesting but I don't think we can integrate it in our clockface because I guess it's not open source. Is it?
Many thanks,
Vito

[Gordon-C]

@vitodefeo 

I've contacted someone on the Device SDK team to respond to this post.  

Gordon

[vitodefeo]

Hi @Gordon-C 
Thanks a lot.... but no-one replied from Fitbit (Google).
Can you please help me?
Many thanks,
Vito

[Gordon-C]

Hi @vitodefeo 

What I understand from our engineering team is what you're trying to achieve is not simple. it's recommend you find a 3rd party developer with existing experience to build their solution.

Anyway, in relation to the specific questions:

1. Can we program the clock face interface in order to save the data to our server? Yes
2. If so, should we use an ftp protocol? No, FTP is not supported, use HTTP POST.
3. Can we, instead, save the data on Fitbit server and download the accelerometer data together with the other data (e.g. HRV) that we download through the Web API? I mean that we use the Web API to download all the data except the accelerometer data and something else (what?) to download the accelerometer data. No, it has to be your own backend.
4. Can the sync performed automatically? Should the user intervene, instead? The device automatically syncs every 15 minutes if the Fitbit mobile application is running on the mobile device, but this excludes accelerometer data.

For the clock face specifically, it would need to do the following:

1. Use OAuth to make the user login with their Fitbit account. This will give the clock face the Fitbit User ID. Example 😞https://github.com/Fitbit/sdk-oauth) << doesn't include handling of refresh tokens, but that probably won't be an issue once they have the User ID.
2. Read accelerometer data Example: (https://github.com/Fitbit/sdk-scientific/blob/master/app/index.js#L40)
3. Periodically send data from the device to the companion, then from the companion to their backend, keyed by the Fitbit User ID. Examples: (https://github.com/dillpixel/fitbit-asap, https://github.com/sammachin/Fitbit-MessageDemo/blob/master/companion/index.js)
4. The clock face would need to check that data is being sent to the backend correctly, and if not, display a message to the user.
   On the backend, you can combine data from the Web API with data from the clock face.

Hope this helps,

[Guy_]

HI @vitodefeo - in light of the Google restrictions for EEA users who will no longer be able to download clocks or apps from the Gallery as of June 2024 it might be wise to look at other brands which have better support for your project in the long term.

The restrictions may not currently apply to the UK [as yet unknown, as no list of countries was specified, so it's open to interpretation] but it may not be long before they too do.

[vitodefeo]

Hi @Guy_ and @Gordon-C ,
Actually, I don't understand this. Why should we download third-party apps?
We need to design our own clock face.
Sorry, I don't understand.
Can you please explain to me more?
Sorry if I don't understand and many thanks,
Vito

[Guy_]

@vitodefeo - for people to install apps on their watch they have to be able to transfer it to the watch from somewhere.

This can be done normally by storing it on the Fitbit online Gallery, in private or public (published) form.

You give the users the link if it is private so they can find it to install. (In published form, if approved, it's available to anyone).

For developers they can load the apps via the developer bridge and the PC development platform. 

The advantage of the private link is it can easily be distributed and installed without having to setup a PC platform or get it approved.

But with the EEA restrictions coming into force in June, the private and public loading may no longer be possible in those countries.

The UK may not be currently included but it may apply to the UK later.

[vitodefeo]

Thanks @Guy_ 
Actually, we can install the app on all the Sense2 before sending them to the participants of our study via post.
So, the users won't need to download anything.
Can this strategy work, in your opinion?

[Gondwana]

I know the question wasn't directed to me, but...

That distribution method would be ideal initially. However, given the cutting-edge nature of the software required, I'd be amazed if bug-fix releases weren't found to be necessary when used for real. Some means of remote upgrading would be required. Side-loading wouldn't be suitable for this, but private GAM links should work (since you're not in EEA).

The alternative would be to post watches back to the developer for upgrades. That would get old fast.

[Guy_]

@vitodefeo - I agree with @Gondwana ,  updating could be a nightmare for any users in countries considered in the EEA, or countries adopting the same EEA regulations in due course, such as the UK, which I believe is underway. How soon that will be may not be known at this point so it is a potential risk for such a project and will need checking.

Preconfiguring watches is also fraught with problems as users adding them to their account for the first time may find they get updated with a new installation. Users must connect them to the Fitbit App account for them to work with your software.

Incidentally, versions of the current V4 release of the Fitbit App aren't as stable as the older release and do lose sync with the watch which may affect timely capturing of complete information. 

Due to bugs or faults that can occur in some watches, users may have to factory reset them and definitely would lose the prepared state in those cases, along with any damaged watches that need replacing under warranty.

Firmware updates can possibly cause issues on this model causing some settings to get lost.

So it may be important to have a guaranteed way of installing the software on watches, done by the users, to avoid administrative complications and surprises over the life of the project.