"Out of memory"-error when processing a large number of images with a macro

memory
java
macro
Tags: #<Tag:0x00007fd540b77858> #<Tag:0x00007fd540b77718> #<Tag:0x00007fd540b775d8>

#1

I’m observing problems when processing a too large number of images using a macro. I get the error <Out of memory>
The code looks somewhat like below.
As mentioned, the problem only occurs when the number of images in the input directory is too large. It seems to me as if ImageJ isn’t clearing the RAM during the individual cycles in the macro.
Does the structure of my macro somehow prevent ImageJ to clear the RAM? From the way my macro works the number of iterations should not be important, because the analysis of the individual images is independent of each other.
Adding
run("Collect Garbage")
only results in the error Unrecognized command: "Collect Garbage"

I’m using ImageJ 1.51m4 on Ubuntu 16.04
Java-Version:
user:~$ java -version openjdk version "1.8.0_131" OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11) OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

input = "/home/user/test/inputfolder/";
output = "/home/user/test/outputfolder/";

function action(input, output, filename) {

open(input + filename);

savename = replace(filename, ".tif", "");

imageId = getImageID();

run("Duplicate...", "title=originalimage");
imageIdOriginal = getImageID();

selectImage(imageId);

// Color Thresholder 1.51m
// Autogenerated macro, single images only!
min=newArray(3);
max=newArray(3);
filter=newArray(3);
a=getTitle();
run("HSB Stack");
run("Convert Stack to Images");
selectWindow("Hue");
rename("0");
selectWindow("Saturation");
rename("1");
selectWindow("Brightness");
rename("2");
min[0]=0;
max[0]=255;
filter[0]="pass";
min[1]=0;
max[1]=255;
filter[1]="pass";
//min[2]=120;
min[2]=70;
max[2]=255;
filter[2]="pass";
for (i=0;i<3;i++){
  selectWindow(""+i);
  setThreshold(min[i], max[i]);
  run("Convert to Mask");
  if (filter[i]=="stop")  run("Invert");
}
imageCalculator("AND create", "0","1");
imageCalculator("AND create", "Result of 0","2");
for (i=0;i<3;i++){
  selectWindow(""+i);
  close();
}
selectWindow("Result of 0");
close();
selectWindow("Result of Result of 0");
rename(a);
// Colour Thresholding-------------


run("Watershed");

run("Analyze Particles...", "size=0-Infinity display exclude clear add");

selectImage(imageIdOriginal);
roiManager("Show All with labels");
run("Flatten");

saveAs("Tiff", output + savename + "_ROI");
saveAs("Measurements", output + savename + ".csv");


close();
}

setBatchMode(true); 
list = getFileList(input);
for (i = 0; i < list.length; i++)
action(input, output, list[i]);
setBatchMode(false);

Thank you for your suggestions!


#2

To find out if this is the case, run your macro once with the batch mode disabled (i.e. setBatchMode(false); instead of setBatchMode(true);), then you should see if some images remain opened. You can add close() statements as required until the macro performs well, then re-enable the batch mode.


#3

Thanks for your answer!
Shouldn’t adding run("Close All"); to the end of the function do the job? When doing ImageJ shows the error “There are no images open.” after it has processed the first image successfully.


#4

Ok, I’ve also tried to close all all remaining open images by adding the following code at the end of my macro (after the last saveAs command), because as @imagejan suspected, indeed I could see images remaining open when running the macro not in batch mode. But this causes my macro to stop after having processed the first image.

macro "Close All Windows" { 
      while (nImages>0) { 
          selectImage(nImages); 
          close(); 
      } 
  } 

I temporarily solved the problem by closing each of the images “individually” via getting the imageID by imageIdOriginal = getImageID(); and then close it at the end of the macro by
selectImage(imageIdOriginal); run("Close")
However, this makes the macro quite unflexible, because with every alteration in a processing step I have to cosider the imageIDs again to ensure all images will be closed in the end.
With my limited experience with ImageJ I don’t quite get why run("Close All"); prevents the macro from working. From my understanding, at the end of each iteration this would close all open images, but the a new image from the array input should be loaded, isn’t it?


#5

Hey Mojo?
I went into EDIT>OPTIONS>MEMORY&THREADS and increased the allocated memory to 8000. Now I can routinely work with 1000 images at once.(sometimes more depending on image size).

You most likely just using up the ram allocated to ImageJ.
But if you increase it too much you can’t run two programs at once.

Hope this helps,
Bob


#6

Yes, that’s exactly the case, but if the macro is programmed correctly this shouldn’t happen, because all the images are processed successively and independent of each other. The RAM doesn’t fill up as long as there are no remaining open images after each iteration of the macro. However, I’m not enough into ImageJ’s macro language so that I understand why run("Close All"); doesn’t work. So I’ll work with the above mentioned temporary solution.


#7

Hi @MojoDodo,

I am having the same issues. My IJ.run (“Collect Garbage”) command at the end of each loop doesn’t seem to do anything, and I still run out of RAM every 4-7 image sets :frowning: