Collecting Real-time Data

[Gondwana]

Because it's a question that often comes up, I'd like to put together a simple demo that does it (which I'll put on github). However, I'm out of my depth in some areas, and would appreciate guidance. Here's the architecture I'm thinking of:

• Watch sends data to companion via Fitbit messaging. I can do this.
• Companion sends data to a server app via fetch(POST) or WebSocket. I've done this using an internet-based server but, for real-time data and personal use, it probably makes more sense for the server to reside on localhost (which would mean phone) or LAN (which would require SSL). I haven't achieved either of those.
• Server temporarily saves data (variable? file? cookie?). This may not be necessary if using websocket to push data to client.
• Server pushes data to a client web page running on localhost or LAN via WebSocket or SSE. Alternatively, client polls server (refresh?), but this is inefficient. node.js seems best choice for websocket support, and many devs will already have this for CLI.
• Client displays data or makes it available for download.

If anyone has any suggestions for a better approach, and/or can suggest appropriate products (eg, free websocket-capable server that can run on localhost phone), please advise!

[JonFitbit]

I did something for an internal project and I used preact and fastify. There's an awesome starter project which will give you a web api, and a web frontend to display some data too. It even comes with a Docker config too.

https://github.com/lmammino/fastify-preact-htm-boilerplate

[Gondwana]

Got a basic system working (see UFO-quality video). 😁

At bottom of video is watch, reporting heart-rate and timestamp.

At left is phone showing companion settings, reporting the values received from the watch (via messaging).

Not shown: custom server running on node.js within Dory on phone, which receives data from companion via WebSocket.

At right is a web page displayed on a desktop PC on the same network, which obtains the data from the server using another WebSocket.

[Gondwana]

Video of accelerometer data streamed in near-real-time.

Data is collected at 60Hz but transmitted in batches of 6 to reduce comms load. The square graph plots the x and y figures; the bar graph plots z.

In addition, the data is made available in CSV format (x,y,z,time), and can be downloaded from a web browser. Here's a snippet:

-2.911,-0.727,9.04,4640
-2.949,-0.057,9.04,4656
-2.241,-0.363,9.098,4672
-2.375,0.057,9.213,4672
-3.064,-0.268,9.27,4688
-3.964,-0.517,9.194,4688
-4.213,-0.536,9.021,4704
-3.715,-0.574,9.079,4720

The time values are occasionally weird, but I think this can be resolved by fine-tuning the batching parameters.

[SahilSS]

Hi can you kindly provide the Github link for the same 

thanks

[Gondwana]

I will, when the code and supporting explanation are in a fit state. That could take weeks; maybe even months.

[EliotJang]

Could you tell your github address?

I failed to search it..

[NouranFlaifel]

Did you post the code anywhere yet? I'm working on a project very closely related to what you are trying to achieve, it would really help. Thanks!

[AntRon]

I am working on a project that uses the Accelerometer and I wanted it to log to a file.  Is there any way to get a copy of the code you used to get the sensor data to display on the app.

Thank you

[Gondwana]

The real-time version was never released. There's a version that transfers in batches in ossapps (Accelerometer Fetcher).

[Walidaljaouni]

Hello can you please help me with my project? i could pay you. thanks