Jerryscript Garbage Collection

[skiddy56]

Is there a way to force garbage collection to run?  It does appear to run when memory gets tight.  From my console log:

[4:28:59 PM]JS memory(11): 61064/65528   (used/max available)
[4:28:59 PM]JS memory(11): 61304/65528
[4:28:59 PM]JS memory(11): 24520/65528
[4:28:59 PM]JS memory(11): 24880/65528

Memory decreased from 61304 down to 24520 in the middle of the routine, presumably because garbage collection ran.  That's great, except that it doesn't always run so timely.  If I load just a 1/3 more data, which should keep me well under 30k (only 7k is data of that 24k), the routine will fail with an out of memory error.  The routine is in a tight loop, moving objects around on the screen.  I can see that each move of an object, increases memory usage.  Hence the increase in memory as above to 61304 but I can also see that the memory can be recovered as evidenced by the decrease to 24520.  This is the same routine that runs out of memory if it starts from a higher memory usage because more data is initially loaded. 

Thanks,

Rich

[EWR]

You can't force the garbage collector but you can do a lot of tricks. For example:

-use short variable names

-clear if you finish your task with it

-try do avoid code duplication

how-to-optimize-javascript-code-bette

https://jscompress.com/  You can try  JavaScript Minifier, you just copy / paste your code and copy it back to your app and it will optimise your code for the low memory.

[skiddy56]

Thanks for the reply.  I was wondering if there were any optimizations to the java script code before being applied to the watch.  From your comments, I guess not.  I'll keep that in mind and I appreciate the links to the optimizations.  But the problem I'm experiencing is not because of the amount of code or the run time.

I tried a change of tact last night and was able to get the application to run although, now it's using between 5 and 10k more data than before. 

Here's the situation:  The application is running through a bunch of on screen objects, executing "document.getElementById" for each one.  The routine calls a method which creates a local variable to hold the element.  This was causing the problem.  Memory continues to increase for each access and is not released in time to avoid the memory problem.  I even tried to set the local variable to null before leaving the method, in the hope that the garbage collection would occur faster.  It didn't.  To fix it, instead of executing getElementById within this routine, I created an array to hold the element's and fill it just once at the start of the program.  I can then reference the array whenever I need the object (rather than executing getElementById each time).  Although it uses a lot more data, it did solve the problem.  Testing different loads (more or less data) I can actually get the base application data up over 55K and it still runs.  Each time the system gets to 63K or so, it runs the garbage collection and gets enough memory to continue.

This got me to thinking why the garbage collection doesn't happen when the getElementById method is part of the main routine.  I'm thinking that there may be events that need to run that have a higher priority than garbage collection.  That's when I found this helpful tidbit:

https://community.fitbit.com/t5/SDK-Development/App-msg-queue-full/m-p/2313147

I'm going to incorporate this into the main routine to see if it works to get the application loaded without creating the array.  Can anyone tell me where this method is documented:  setTimeout(continueWorkFunction, 1)?

Thanks,

Rich

[EWR]

setTimeout(continueWorkFunction, 1) is just a timeout on your functions to give it a little time to process. You can call any of your functions in setTimeout(anyFunction, 1).

On document.getElementById you can try to make an optimisation.

Instead of declaring variables

x = document.getElementById('1');

y = document.getElementById('2);

You can parse the HTML DOM using nodes. Like

x = document.getElementById('1');

x.nextSibling to get the y value

You will see some improvement using this method. It's better than saving.

Good luck,

Eduard

[skiddy56]

Thanks, I will give that a try as well.  

To those who are interested, the setTimeout function is a javascript function.  I was a little confused by the example in the post:https://community.fitbit.com/t5/SDK-Development/App-msg-queue-full/m-p/2313147

The syntax:

setTimeout(updateDisplay.bind(this, i + 1), 1);

didn't seem to match to :setTimeout(continueWorkFunction, 1)

A little research later, I found this: https://stackoverflow.com/questions/10312963/difference-between-settimeout-with-and-without-quotes-a...

on Stack overflow, always a good resource.  The next to last answer describes the usage of .bind.

[skiddy56]

Hi Eduard,

I wanted to get back to you to let you know I tried the approach you suggested to use the nextSibling.  Unfortunately it had little or no effect on memory.  It was a good try, I really thought it might work. 

So far the best solution is to load the elements into an array first and use the array during the initial load.

I can't say why, but the problem is always when I have a file open on the watch and I'm executing the getElementById method while it's open.  If I move the getElementById method so I'm not calling it while the file is open, everything works fine.  Memory still increases with each call but the garbage collection runs and recovers the memory allowing the process to continue.

I don't know why having the file open should make a difference.  I'm reading just 128 bytes at a time.  I started at 1024 but kept reducing to try and fix the memory problem.

Thanks,

Rich

[EWR]

Ok, so you are reading a file on the watch, right? I didn't get this before. What type of file are you reading? Is it a JSON? Are you fetching it from the companion and sending it to the watch?

[skiddy56]

Sorry, I didn't mention it before...  I am reading a file.  The file is downloaded via the resources (for now).  Seemed the easiest way to get it there while testing.  It is not a JSON file.  It's straight 8 byte ascii. 

The routine to manage the file is a javascript object.  The constructor for the object opens the file and sets up for the first read:

var FS= fs.openSync(FileName,"r");

var buffer = new ArrayBuffer(BufferLength);
var Data;
var DataString;var BufferLength=128;

var CurrPosition=0;

There's a method within the object GetNextRecord() which will return the next logical record.  If need be, the next buffer is read from the file:

BytesRead=fs.readSync(FS,buffer,0,BufferLength,CurrPosition);

if (BytesRead < buffer)
          {
            DataString=String.fromCharCode.apply(null, new Int8Array(buffer.slice(0,BytesRead)));
          }
          else
          {
            DataString=String.fromCharCode.apply(null, new Int8Array(buffer));
          }

 }

The DataString is parsed and the next "record" returned to the calling function which will then call it again until the end of the file is reached.

I should mention that I do close the file at the end:

fs.closeSync(FS);

Rich

[skiddy56]

I just found a mistake in the previous post.  The line:

if (BytesRead < buffer)

should be:

if (BytesRead < BufferLength)

[EWR]

Does the file contain anything you don't need to read? Try to free up the file a little if it's possible for the beginning. The problem is that when you open the file it will load the memory and it will not be cleaned until you finish with it. Have you tried to save everything from the file in a variable?

  let FS = fs.readFileSync("FileName", "r");

and you will have all your data from the file in that variable. Maybe this would help you a little. 

[skiddy56]

I'm not sure that I understand your statement:

"The problem is that when you open the file it will load the memory and it will not be cleaned until you finish with it"

I'm not sure that I follow.  The reason I'm doing a buffered read is to reduce the amount of memory needed to process the file.  At the current buffer length of 128 bytes, all I need to process the file are the 128 bytes, some code memory and some system memory to hold the file open. 

Are you saying that the system memory needed to hold the file open and perform the read is really high? 

Thanks, Rich

[Gondwana]

I can think of an appallingly bad thing to try: reduce use of local variables, especially in frequently-called functions or loops. Every time a local variable is declared, additional memory will be set aside for it. Yes, it should be garbage-collected before Bad Things happen, but if GC isn't responsive enough...

A largish buffer in a loop or oft-called function that reads from a file could be a good candidate.

[EWR]

Opening a file and keeping it open will load the js memory. From what I know the GC can't clean the open file until you close it or stop using it. That's why I would try to read it once and save everything in a variable. Maybe I am wrong 😞

Hope this is working and can improve the us memory size. 

[skiddy56]

Thanks for the input.  You're exactly right.  It's one local variable that's created during a loop that occurs 296 times.  It's the allocation of memory to get an element using getElementById.  Each loop increases the memory usage and continues to increase until either the garbage collection runs or the system runs out of memory.  I tried declaring the variable as a global variable but that didn't work.  Thinking about it, I realized of course that didn't work, the getElementById method is returning a new object each time, regardless of where it's declared. 

The interesting part of this is the problem only occurs if the routine is calling getElementById during the loop AND the file is open (and being accessed).  If I change either condition, GC runs and everything works. 

My temporary solution is to create an array to hold the elements before the routine runs.  This way I just reference the element from the array, no need to call getElementById during the routine.  This works.  I can see memory is still increasing with each call, but garbage collection always runs just in time to recover the memory.

[skiddy56]

I can't predict how large the file will be so reading the whole thing into memory is risky.  I think for now, I'll keep with the solution to use the array to hold the elements.  I realized I can delete the array once the routine runs so I'll only need it during the initial load.

Thanks for all the suggestions.

One last note (unless I find a solution that doesn't require the array): The routine to read the file is reading in a buffer of 128 bytes and then copying that to a string of 128 bytes.  I wonder if that 256 bytes, created during the routine, is enough to push the memory over the limit?  I wonder if anyone can tell us how close to the limit does the usage have to get before garbage collection is called?