Batch mode exception error


#1

Hello
I’m writing macros to run in the batch mode. I keep getting an error when running my macro in the Batch mode. I purpose of this macro is to open all the composite images in a directory and rename the file based on the original file name and the number of green foci.

(Fiji Is Just) ImageJ 2.0.0-rc-61/1.51n; Java 1.8.0_45 [64-bit]; Windows 7 6.1; 68MB of 2954MB (2%)

 java.lang.NullPointerException
at ij.gui.ImageWindow.close(ImageWindow.java:397)
at ij.gui.StackWindow.close(StackWindow.java:192)
at ij.ImagePlus.close(ImagePlus.java:390)
at ij.CompositeImage.close(CompositeImage.java:624)
at ij.plugin.BatchProcessor.processFolder(BatchProcessor.java:232)
at ij.plugin.BatchProcessor.run(BatchProcessor.java:110)
at ij.IJ.runPlugIn(IJ.java:187)
at ij.Executer.runCommand(Executer.java:137)
at ij.Executer.run(Executer.java:66)
at java.lang.Thread.run(Thread.java:745)

Can anyone tell me what these errors means? Thanks in advance.


#2

This is the macro code that I am working with.

// macro 2
// input, dir with mulitple cropped red 'SC' files 
//output, renamed files that were made into blobjects -> 1CO, ect

setBatchMode(false);
if(endsWith(getTitle(), "SC.tif") ){
//filename = getDirectory( getTitle() );
filepath = getDirectory("image"); // getDir opens up the window
filename = getInfo("image.filename");//this gets me what i wanted
filename_indx = substring(filename, 0,1);
//mainTitle_path = substring(mainTitle,  0, lengthOf(mainTitle)-5);
print(getTitle()+"  sc. proceeding with image");
//FIRST CODE BLOCK preping Image and segmentation. Remove previous scales, split and name channels
//test for size
width = getWidth;
height = getHeight;
if(width * height < 100){
	print("too small, should be deleted");
}else{
print("size test passed");
run("Set Scale...", "distance=0 known=0 global");
T = getTitle;
selectWindow(T);
run("Duplicate...", "title=duplicate duplicate");
selectWindow(T);
run("Stack to Images");
selectImage(2);
redImage = getTitle();//this is not the same as renaming the image?
selectImage(3);
greenImage = getTitle();
selectImage(4);
blueImage = getTitle();
imageCalculator("and", greenImage, redImage);
setAutoThreshold("Shanbhag dark");
setOption("BlackBackground", true);
fociChannel = getTitle();

print("channels seperated");
//First channel, blue channel. Segment and add centromeres to RoiManager.
selectWindow(blueImage);
run("Threshold...");
//user input needed here...
waitForUser("Step 1, Thresholding", "Adjust threshold for centromere signal then press OK" );
run("Convert to Mask");
run("Despeckle");
run("Dilate");
run("Analyze Particles...", "  show=Nothing exclude add");
IJ.redirectErrorMessages()
wait(500);

for(i=0;i<roiManager("count");i++){
	roiManager("select",i);
	cIndex = i+1;
	roiManager("Rename", roiManager("index") + "_centromere "+ cIndex);
	Roi.setProperty("obj_class", "centromere");
	Roi.setProperty("cent_indx", i);
	roiManager("update");
	roiManager("select",i);
}
var centromereCount = roiManager("count");
var blobCount = 0;

selectWindow(greenImage);
run("Threshold...");
waitForUser("Step 1, Thresholding", "Adjust threshold for foci signal then press OK" );
run("Analyze Particles...", "size=4-Infinity exclude add");//size=6 also works well
var fociCount = 0;
for(f=centromereCount;f<roiManager("count");f++) {
	fociCount++;
	roiManager("select", f);
	roiManager("Rename", roiManager("index")+ "foci "+ fociCount);
	roiManager("update");
}//end green channel processing
print("foci added");

//Third channel, Red channel. Running Ridge detection (RD) to 'detangle' skeleton. Adding 'ridges' to RoiManger
selectWindow(redImage);
run("Threshold...");
waitForUser("Step 1, Thresholding", "Adjust threshold for foci signal then press OK" );
run("Convert to Mask");
run("Despeckle");
run("Invert LUT");//inverting LUT required for correct RD performance
// Requires the Ridge Detection plugin from the Biomedgroup update site.
//run("Ridge Detection", "line_width=2 high_contrast=255 low_contrast=240 add_to_manager");
run("Ridge Detection", "line_width=2 high_contrast=255 low_contrast=240 extend_line show_junction_points show_ids displayresults add_to_manager method_for_overlap_resolution=SLOPE sigma=1.2 lower_threshold=16.83 upper_threshold=40");
//sigma 1.2, lower 16.83, upper 40

var scCount = 0;
var JPcount = 0;
for(o=0;o<roiManager("count");o++){
	roiManager("select",o);
	if (startsWith(Roi.getName(), "C")) {
			roiManager("Rename", roiManager("index") + "SC ");
			scCount++;
			}
	if (startsWith(Roi.getName(), "JP-")) {
		o--;
		}
}
print("sc processed");

for(cen=0; cen < roiManager("count");cen++){
	roiManager("Select",cen);
	if(matches(Roi.getName(), ".*centromere.*")) {
		roiManager("Select",cen);
		makeBlob(cen);
		Roi.getCoordinates(centx, centy);
		roiManager("update");
		blobCount++;
		}
		}// fociCount+1 should be sc
		}//size test else

var ObjClass = "_CO" + fociCount;
selectImage(5);
close();
selectImage(4);
close();
selectImage(3);
close();
selectImage(2);
close();

selectImage(1);
mainTitle=getTitle();
mainTitle_path = substring(mainTitle,  0, lengthOf(mainTitle)-5);//remove tiff

input = "C:\\Users\\alpeterson7\\Desktop\\macro material\\foofoo\\test2\\";

print("going to rename "+ filename+" to "+ filename_indx+ObjClass);
saveAs("Tiff", input + filename_indx+ObjClass +".tif");

roiManager("reset");
selectWindow("Results");
close();
run("Close");

}else{
	filename = getInfo("image.filename");
	print(filename+" skip this file, not SC");	
	close();
}
//roimanager ends up being opended... but doesn't seem to open big image

//figure out two or one blob
function makeBlob(cen){
	print("entering makeblob");
	roiManager("Select", cen);
	Roi.setProperty("reverse", "no");
	print("getting centromere coordinates"); // maybe just switch to roi is in function
	Roi.getCoordinates(centx, centy);
	print("a coord "+ centx[2] + " " + centx.length);
	roiManager("deselect");
	for(noncen=centromereCount+fociCount; noncen < roiManager("count"); noncen++) { // make sure these counters will work in function format
		print("selecting noncen " + noncen);
		roiManager("Select", noncen); //select roi to test if SC
		if(matches(Roi.getName(), ".*SC.*")) { 
			print("name matches sc ");
			Roi.getCoordinates(SCx, SCy);
			print("getting SC coordinates");
			print("a coord "+ SCx[2]);
			roiManager("deselect");
			paired=false;
			for(k=0; k < centx.length && !paired; k++){  //k pixels in centromere
				print("in centromere pixel loop");
				roiManager("Select", noncen);
				if(Roi.contains(centx[k], centy[k])) { // this test is for if SC runs through centromere
		 				print("test if SC is in centromere");
		 				roiManager("deselect");	
		 				roiManager("Select", cen);
						if(Roi.contains(SCx[2], SCy[2])) {  //change this to [5], since some ends of SC extend past centromere
								print(cen + " " + noncen + "SC array starts at centromere");
										}									
								if(Roi.contains(SCx[SCx.length-1], SCy[SCy.length-1])) { //if cent contains end of SC
									print(cen + " " + noncen + "centromere is at end of array. Reversing Arrays");
									SCx = Array.reverse(SCx);//
									SCy = Array.reverse(SCy);
									Roi.setProperty("reverse", "yes");//set for centromeres
									roiManager("update");
									}	 					
		 						run("Measure");
								length1 = getResult('Length', nResults-1);							
								SClength = Roi.getProperty("SC Results length");//SC Results Length		 												
		 						print("the SC length from the SC is "+SClength);
								roiManager("update");										 						
		 						blob_parts = Array.concat(cen, noncen);//noncen = SC
		 						roiManager("Select", blob_parts);//roiManager("Select", newArray(cen, noncen)) doesn't work
		   						roiManager("Combine");
								roiManager("Add");//new object
								roiManager("deselect");
								roiManager("Select", roiManager("count")-1); //select the new roi
								roiManager("Rename", roiManager("index")+"_blob");
								print("blob made");
								Roi.setProperty("SC_Results_length", length1);
								Roi.setProperty("SC_array_length", SCx.length); // SC array length  
								Roi.setProperty("SC_index", noncen);																
								Roi.setProperty("obj_class", "blob");							
								roiManager("update");
								noncen=1000; //break out of loop by overcounting 
								blobCount++;
								paired=true;
					        	}
			            }//k pixels in centromere   	
		         }//noncen name matches SC
			}//start cycling thru SCs 
}//end function

#3

They are related to your close commands. Since you work in batch mode, no window is displayed so there is no need to close them. Removing the close calls should make it work.


#4

This is the macro code that I am working with.

setBatchMode(false);


The only mention of batch mode in the macro is at the beginning where you set it to 'false'. Is this meant to be 'true'?

#5

@iarganda on this topic, in cases where you do use close() commands in a batch mode macro, does it cause a catastrophic error to halt the macro? Or will the macro still complete, but these are sort of ‘warnings’?

I use close() commands to close intermediary windows in a batch mode macro that exits batch mode halfway using setBatchMode("exit and display") (setBatchMode(false) isn’t used in this case because I want more than just the active window to be shown). I was under the impression that this would speed up the macro because the intermediary windows would no longer be displayed upon batch mode exit (assuming it takes time to render them?).

Could you comment on this, as to whether it is beneficial to use close() commands in the example above in batch mode?


#6

Thank you for the reply. That is helpful. I adjusted the batch mode and close() lines in my script. Now Batch Mode will process all of the files within the selected folder. There are 3 issues that still need a little work.

  1. I am having an issue with setting the threshold in batch mode. I get different results in batch mode ‘run(“Threshold…”, “Minimum”);’ (6 particles) than when I set the threshold “by hand” outside of batch mode (2 particles, the correct number). Why would run(“Threshold”) be performing differently in batch mode?

  2. Inverting the LUT and running Ridge Detection still pop up windows requiring pressing ‘ok’. Is there a way to turn it off?

  3. Batch mode opens ROI.zip file (made in a previous step) and this throws an exception. Is there a way to skip over/not open the ROI.zip folder in batch mode?


#7

When you say “by hand” are you setting the threshold to method = “minimum”? Or setting the upper/lower threshold by hand? Just to confirm, are you ensuring that when you record your manual thresholding with the macro recorder, that no other settings are recorded? And are you thresholding at the same point as the macro is (i.e. no further processing has been done to the image in the macro before thresholding commences)?

To debug your macro you can copy the macro script in to a text window (Plugins->New->Text Window…) and use the debug menu to step through it and figure out if it’s doing as you would expect if you did it manually.

When does this ROI.zip folder get opened in the script, I can’t locate any part of the script where a .zip file is opened? Maybe try running your script through debug to locate that also, and you may be able to figure out how to prevent it.


#8

Thanks for the debug suggestion. I’ll try working with that. Hopefully it can serve some use in the thresholding issue.

When you say “by hand” are you setting the threshold to method = “minimum”? Or setting the upper/lower threshold by hand?

By hand means just choosing ‘Minimum’ from the algorithm list. I’m not adjusting other things before and it is at the same point that the macro would be.

When does this ROI.zip folder get opened in the script, I can’t locate any part of the script where a .zip file is opened? Maybe try running your script through debug to locate that also, and you may be able to figure out how to prevent it.

The folder that the macro is running on contains a Roi.zip folder (made from a previous macro).


#9

Are you running this macro from the FIJI script editor or from Process->Batch->Macro? I can’t see an open() command at the start of the script, so are you opening the image before running the script?


#10

I am running this macro in FIJI from Process->Batch->Macro (Process). From my understanding the batch mode opens each file within a directory so open() would not be needed. Is that correct?

I think I fixed the Thresholding issue. I changed my lines of
run("Threshold...", “Minimum”) to setAutoThreshold("Minimum dark")'

I looked back at what is displayed in the Recorder window while setting threshold values. What is the difference between setAutoThreshold() and run("Threshold...", "Minimum")? Both seem to work in batch mode, but setAutoThreshold() seems to behave more predictably.

One of the remaining issues is that is remaining is that LUT inverting and Ridge Detection show windows even with setBatchMode("hide"); That might go to another post.

Thanks!


#11

You seem to be doing pretty well with the macro scripting, I would look at using the script editor to do all of the work and not use the Process->Batch->Macro method, see this page for details. It gives an example of running a blank function through a whole folder. From there you can use your script (below) to only run through .tif files and avoid any other files.

if(endsWith(getTitle(), "SC.tif") ) {

Although, I’m not sure why that currently isn’t working in the built-in batch processor, it shouldn’t be opening your .zip files as the if statement encompasses your entire script. Maybe try:

if(endsWith(getTitle(), "SC.tif") >0) {

Sorry I can’t help you with the rest of your problems, I don’t know the differences between the two pieces of threshold code. When browsing the documentation, they seem to be the same algorithms. When I record myself doing it the code is:

setAutoThreshold("Minimum dark");

EDIT: Running run("Threshold...", "Minimum"); results in the “threshold” window coming up with the minimum threshold preview, but doesn’t actually set anything, so maybe it doesn’t set the threshold to minimum for when you run("Convert to Mask");, it probably uses the default thresholding, resulting in different particle analysis later down the line. Try doing default threshold and analyze particles, see if you get 6 particles?


#12

I played a bit with it and I noticed that

  1. the close() command does not through an error if an image is open (even in batch mode), but
  2. if it does because no image is open and you try to close, then yes, it halts the macro.

#13

I have come across the processFolder() macro function before (Easy vs Flexible) – I haven’t been to eager to work the 'Flexible" option of Batch processing into my image analysis because the script I have been working on ballooned into a very large macro. I’ve found that the “Easy” option of Batch processing provides a good incentive to keep my macros simple and small.

This macro that I am running in Batch mode is a simpler version of a >1000 line macro I’ve been working on with the script editor for about ~2 years (previous posts [here](Creating object class with rois from multiple channels) and [here] (Manually refining detected ridges) ). I learned alot and wrote some complex functions, but the script became too detailed oriented in terms of improving the accuracy of the measurements for each image I was trying to extract. Processing each image with my macro started taking ~20 min to give me the results, which was not feasible for my data set. I switched to running 3 macros in batch mode from 1 giant macro because I decided that taking the best measures from my data was much faster than making sure all of the measurements were perfect for each image.

My current plan for smaller macros within Batch
macro1 (apply to a folder with cell images), output new folders for each cell image with smaller images of segmented shapes cropped from the larger image) (*DONE)
macro2 (apply to the segmented shapes in each new folder of macro1). output: rename the shape images if they meet criteria, or move the images to a new ‘discard’ folder if not. (*UNDER CONSTRUCTION)
macro3 apply to the same folder as macro2, print out measures of the ‘passing’ shape images. (*UNDER CONSTRUCTION)

I would appreciate hearing more of the benefits of nesting my macros in the ‘flexible’ processFolder() instead of the 'Easy" Process->Batch->Macro (Like will be easier to run from cmd line?)


#14

I’m a novice (1-2 years experience writing macros in IJ1 macro language only), I just assume that it’s more flexible to use your own recursive batch processing. I don’t really use that processFolder function from the link. But I make a file name list from a user-defined folder and work with that.

dir = getDirectory();
array = getFileList(dir);
if (endsWith(array[i], ".tif") {
[statements here]
}

That’s a rough outline of what I do.