Cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Logging raw data using filesystem API

ANSWERED

Hi,

I want to log HR, Gyro and Accelerometer data during sleep to a file and then transfer it later when connected.

I'm getting the data OK it's just the writing to a file I'm struggling with. I've got a button that starts the recording every second and then stops the recording and shows the file stats to the console. I'm opening a file in append mode and want to add each 1 second of data to the file.

When I press the stop button the stats always show 40 bytes written!

The writing  to file occurs in the refreshData() function. I think it's got something to do with me not understanding the ArrayBuffer and maybe not converting to binary beforehand? I couldn't find any examples anywhere of how to append data to a file.

 

import document from "document";
import { Accelerometer } from "accelerometer";
import { HeartRateSensor } from "heart-rate";
import { Gyroscope } from "gyroscope";
import * as fs from "fs";

let thisDate = new Date();
let recFilename = `RawDataLogger-${thisDate.getFullYear()}${("0" + (thisDate.getMonth() + 1)).slice(-2)}${("0" + (thisDate.getDate() + 1)).slice(-2)}.txt`;
console.log(recFilename);
let fileID = 0;
//let buffer = new Uint8Array(8);
let buffer = new ArrayBuffer(8);

let accel = new Accelerometer();
let hrm = new HeartRateSensor();
let gyro = new Gyroscope();

// let accelData = document.getElementById('accel-label');
// let hrmData = document.getElementById('hrm-label');
// let gyroData = document.getElementById('gyro-label');
let btnRecord = document.getElementById('btn-rec');

let recording = false;
let recFunction = null;

btnRecord.onclick = function(evt){
  if (recording) {
    recording = false;
    btnRecord.text = "Record";   
    
    // Stop Device Sensors
    accel.stop();
    hrm.stop();
    gyro.stop();
 
    // Cancel the recording function
    clearInterval(recFunction);
    
    // Dev Check - Check the stats of the file written
    let stats = fs.statSync(fileID);
    if (stats) {
      console.log("File size: " + stats.size + " bytes");
      console.log("Last modified: " + stats.mtime);
    }
    // Close the file
    fs.closeSync(fileID);
  } else {
    recording = true;
    btnRecord.text = "Stop";
    
    // Start Device Sensors
    accel.start();
    hrm.start();
    gyro.start();
    
    // Init the file
    fileID = fs.openSync(recFilename, "a+");
    
    // Set callback recording function
    recFunction = setInterval(refreshData, 1000);
  }
}

function refreshData() {
  // Populate ArrayBuffer
  buffer[0] = thisDate.getTime();
  buffer[1] = hrm.heartRate ? hrm.heartRate : 0;
  buffer[2] = accel.x ? accel.x.toFixed(1) : 0;
  buffer[3] = accel.y ? accel.y.toFixed(1) : 0;
  buffer[4] = accel.z ? accel.z.toFixed(1) : 0;
  buffer[5] = gyro.x ? gyro.x.toFixed(1) : 0;
  buffer[6] = gyro.y ? gyro.y.toFixed(1) : 0;
  buffer[7] = gyro.z ? gyro.z.toFixed(1) : 0;
  
  // Write ArrayBuffer to file
  fs.writeSync(fileID, buffer);
}

 

 I could correctly read and write using a readFileSync & writeFileSync but I need to keep appending each set of data which I assume means usig writeSync.

Any help or pointers anybody please?

Best Answer
0 Votes
1 BEST ANSWER

Accepted Solutions

Hi,

I haven't tested this but you should close the file from time to time, to flush any cached data.

Basically I would create a function like this and call it every time I want to add something.

 

function append(filename, buffer) {
  let fd = fs.openSync(filename, 'a+');

  fs.writeSync(fd, buffer);

  fs.closeSync(fd);

}

 

https://github.com/kmpm | Using: Ionic | Retired: Alta

View best answer in original post

Best Answer
0 Votes
7 REPLIES 7

Hi,

I haven't tested this but you should close the file from time to time, to flush any cached data.

Basically I would create a function like this and call it every time I want to add something.

 

function append(filename, buffer) {
  let fd = fs.openSync(filename, 'a+');

  fs.writeSync(fd, buffer);

  fs.closeSync(fd);

}

 

https://github.com/kmpm | Using: Ionic | Retired: Alta
Best Answer
0 Votes

THANK YOU!! 

I thought I'd tried to do a close after every write, but maybe my code was in a different state and something else was wrong at the time. I was also worried about doing it every second, but it seems ok.

Brilliant! I can crack on now, tidy it up and then get a companion app to transfer the file to.

 

Thank you again.

Best Answer
0 Votes

You could keep the file open for a while, for several writes, and then do a close+reopen.

The following example MIGHT try to close the file every 5 seconds. It will reopen if it is closed using the getFd() method. If someone closes the app before you do a close I guess you might loose that data. You would have to test it.

 

var _fd = null;

const filename = 'somefile.txt';

 

setInterval(closeFd, 5000);

 

function closeFd() {

  if (_fd != null) {

    fs.closeSync(_fd);

    _fd = null;

  }

}

 

function getFd() {

  if (_fd === null) {

    _fd = fs.openSync(filename, 'a+');

  }

  else {

    return _fd;

  }

}

 

function append(buffer) {

  let fd = getFd();

  fs.writeSync(fd, buffer);

}

 

https://github.com/kmpm | Using: Ionic | Retired: Alta
Best Answer
0 Votes

Yeah that looks like it'll do the job, a singleton that last for 5 secs and respawns!! I'll test tomorrow.

 

I was gonna look into making it a background app anyway so there shouldn't be an issue with losing data. I'll leave the code open/closing every second for tonight and see what kind of battery life it uses. Every 5/10 secs I expect would improve battery life and I might not even need 1 second granularity anyway, but the more data a learning algorithm has the better chance it has of spotting a pattern.

 

Cheers again!

 

Best Answer
0 Votes

Hook into the Appbit onunload event to close the file if still open. (https://dev.fitbit.com/build/reference/device-api/appbit/)

 

Belt and braces and all that. 🙂

Best Answer

Cheers @SunsetRunner I like me some belt 'n braces! I don't mind a bit of code bloat if it's for good reason! Besides it's good to know so I can incorporate onunload in a std app template. As I learn more I enhance my template adding more useful things. I'll actually make a post if people can share what they use to see if we can come up with a good one.

 

I ran my logger for 8 hrs last night and it used 12% battery. I'm gonna put in a 5 sec interval buffer write instead of every 1 sec to see if I can claw back 1 or 2 % of that juice. I'm under no illusions that it'll be the 1 sec readings that use the bulk but it'll be interesting to see if it saves any at all.

 

Best Answer
0 Votes

Hi, this is an old thread. However, I'm wondering if you can share the whole app code? I'm interested in recording raw accelerometer data, but am having trouble combining it with the file transfer code. I'm not very good at JS so some references would be very helpful!

Best Answer
0 Votes