Batch Processing with Generic File Names

batch-processing
macro
Tags: #<Tag:0x00007fd54687b018> #<Tag:0x00007fd54687ac08>

#1

Hi All,

Super new to writing macros and have learned a bunch from this forum already. I am trying to perform the following workflow on a folder of images:

  1. Open Original
  2. Copy Original, make new, identical image.
  3. Change threshold on New Image.
  4. Convert to Mask.
  5. Divide by 255.
  6. Use Image Calculator to multiply ORIGINAL image times NEW IMAGE.
  7. Save "Result of " image in new folder.

Below are the commands I have from the recorder tool (with the original image already open):

open("ORIGINAL.tif");
selectWindow("ORIGINAL.tif");
run("Select All");
run("Copy");
newImage("NEW IMAGE", "8-bit black", 1360, 1024, 1);
run("Paste");
setThreshold(20, 255);
setOption("BlackBackground", false);
run("Convert to Mask");
run("Divide...", "value=255");
selectWindow("ORIGINAL.tif");
imageCalculator("Multiply create", "ORIGINAL.tif","NEW IMAGE");
selectWindow("Result of ORIGINAL.tif");
saveAs("Tiff", "C:\\Users\\Result of ORIGINAL.tif");
close();

Is this workflow straightforward enough to use the batch processing tool? If so, how do I go about making generic file names in the code so it can perform the necessary operations within each loop? I’ve read the “How_to_apply_a_common_operation_to_a_complete_directory” article and have been unsuccessful so far. I would need to change the file name at every line where ORIGINAL currently is, in addition to making ImageJ save the result with a filename specific to the original (right now it spits it out as “Result of ORIGINAL” just using the given filename and it would be great to keep this convention).

Is it better to use the Script Editor with the provided Process Folder template? If so, what are the correct changes to the code to properly process the folder? The folder contains 75 images and their filenames are all identical save for a sequential number change (Original1, Original2, Original3, etc).

Thank you so much!


#2

Hi @velivel

I don’t know about the batch processing tool, but the script editor you can find Templates > ImageJ 1.x > Examples > Process Folder (IJ1 Macro). There you can copy paste your code in the processFile() function.

Then you will have to change the following lines:

open("ORIGINAL.tif");
...
selectWindow("ORIGINAL.tif");
imageCalculator("Multiply create", "ORIGINAL.tif","NEW IMAGE");
selectWindow("Result of ORIGINAL.tif");
saveAs("Tiff", "C:\\Users\\Result of ORIGINAL.tif");

The input and output variables are managed by the parameter feature of the scripting tools


The processFolder also gives you the file variable containing the filename.
All the references to the ORIGINAL.tif have to be replaced with the file variable now. This would look something like this:

open(input + file);
...
selectWindow(file);
imageCalculator("Multiply create", file, "NEW IMAGE");
selectWindow("Result of " + file);
saveAs("Tiff", output + file);

Note that input + file is a string concatenation of the input-folder and the filename.


#3

Thank you so much! I have gone through and made the necessary edits to the Process Folder template, however, when I go to run I get the following error: “File is not in a supported format, a reader plugin is not available, or it was not found”, then it shows me the pathway to the folder I am designating my input. All my files in this folder are .tif, as are the files I am exporting to the new folder. Is there something additional I need to modify in the processFolder() function as well as your given edits to the processFile() function?

Again, thank you so much!


#4

@velivel check the console and the log window to exactly pinpoint the location of the error.
You can use IJ.log("the value of my variable is: " + file); to check what you are trying to open.

In some cases of compressed tif files imagej does not always manage to decode them. You can test this by trying to manually open one of the files in the input directory (but since you managed to record the commands, and hence did that already, this is porbably not it)

My guess is that there might be other files of different format than tif in the input directory that the command ’open(input + file) cannot open.


#5

Awesome! Just ran IJ.log("the value of my variable is: " + file); and the script editor returned “undefined variable in line 12”. In my code, this is the line that reads:

processFolder(input);

It is the beginning of the whole macro. Do I need to make further edits to the process folder function? thanks again!


#6

For reference, this is the full text of what I am working with:

processFolder(input);

function processFolder(input) {
	list = getFileList(input);
	for (i = 0; i < list.length; i++) {
		if(File.isDirectory(input + list[i]))
			processFolder("" + input + list[i]);
		if(endsWith(list[i], suffix))
			processFile(input, output, list[i]);
	}
}

function processFile(input, output, file) {
open(input + file);
selectWindow(file);
run("Select All");
run("Copy");
newImage("Untitled", "8-bit black", 1360, 1024, 1);
run("Paste");
setAutoThreshold("Default");
//run("Threshold...");
setThreshold(20, 255);
//setThreshold(20, 255);
setOption("BlackBackground", false);
run("Convert to Mask");
run("Divide...", "value=255");
selectWindow(file);
imageCalculator("Multiply create", file, "Untitled");
selectWindow("Result of " + file);
saveAs("Tiff", output + file);
close();

print("Processing: " + input + file);
print("Saving to: " + output);
}

#7

Any more suggestions? The macro is still returning “File is not in a supported format, a reader plugin is not available, or it was not found” followed by the name of my Input Folder and the name of one of my images. However, the name of the image in the error box displayed changes sequentially when I delete the image supposedly giving the error. Is there a typo or a poorly defined variable somewhere in my processFolder function?


#8

@velivel, could you paste the entire macro that you are using? I think some parts may be missing above because processFolder(input); is line 1, not line 12.

When you run the macro do you get prompted to choose input and output folders? And are the folders separate (not nested)?

It seems like the original and duplicate images are left open after the processing, and this can cause problems. You should select each window in turn and close(); it.

As @FelixM mentioned, non-image files can cause problems too – e.g. I had a batch macro that was broken by a custom icon file added by Google Drive. I used the command line (Terminal on Mac) to view and remove the invisible files. Presumably there’s a way to do this on Windows too.


#9

@tswayne

Hello!

This is the full macro, I am using the ProcessFolder.ijm template found in the Script Editor of Image J:

// @File(label = "Input directory", style = "directory") input
// @File(label = "Output directory", style = "directory") output
// @String(label = "File suffix", value = ".tif") suffix

/*
 * Macro template to process multiple images in a folder
 */

// See also Process_Folder.py for a version of this code
// in the Python scripting language.

processFolder(input);

// function to scan folders/subfolders/files to find files with correct suffix
function processFolder(input) {
	list = getFileList(input);
	for (i = 0; i < list.length; i++) {
		if(File.isDirectory(input + list[i]))
			processFolder("" + input + list[i]);
		if(endsWith(list[i], suffix))
			processFile(input, output, list[i]);
	}
}

function processFile(input, output, file) {
open(input + file);
selectWindow(file);
run("Select All");
run("Copy");
newImage("Untitled", "8-bit black", 1360, 1024, 1);
run("Paste");
setAutoThreshold("Default");
//run("Threshold...");
setThreshold(20, 255);
//setThreshold(20, 255);
setOption("BlackBackground", false);
run("Convert to Mask");
run("Divide...", "value=255");
selectWindow(file);
imageCalculator("Multiply create", file, "Untitled");
selectWindow(file);
close(file);
selectWindow(Untitled);
close(Untitled);
selectWindow("Result of " + file);
saveAs("Tiff", output + file);
close();
	
}

Once I run the macro it does prompt me to select input and output folders in a separate dialog window, and once I choose them it gives me the error I mention above. The folders are separate! Additionally, I have gone through and added commands to close the original and duplicate, as suggested. I have tried this macro on both Windows and Mac and I get the same error message for both. Additionally, I have tried using a different, random folder of images to see if maybe my images were corrupted in some way and received the same “File not in a supported format/or reader plugin missing/or folder not found” error. I’ll try to use Terminal to see if there’s any invisible files. Thank you in advance for your help!


#10

Dear @velivel,

The paths returned by the script directive (see Fiji Scripting directive misbehaves on Windows for more details on the topic) do not contain trailing slashes/backslashes. Therefore, you have to replace the quoted line with open(input + File.separator + file) to open a correct path.

Best,
Stefan


#11

Hello,
I am completely new to writing any type of macros/scripts in Imagej and although I would like to learn more, I’m eager for this script to move on with data processing.

I’d like to subtract a background image from all images in a file folder, which have sequential numeric suffix.

The code follows as above, however using only the imageCalculator and invert functions. The background image is in the same folder as ‘input’, but I keep getting an error at line 34 image not found.

Any ideas what the issue is?

function processFile(input, output, file) {
	// Do the processing here by adding your own code.
	// Leave the print statements until things work, then remove them.
	print("Processing: " + input + file);
	imageCalculator("Subtract create", "backgroundimage.tif", input + file);
	selectWindow(file);
	run("Invert");
	selectWindow("Result of intput + file");
	saveAs("Tiff",output + file);
	close();
	print("Saving to: " + output);
	
}

#12

You have to open the image before this line since the arguments to imageCalculator are titles of open images (or IDs), not paths to images:

open(input + File.separator + "backgroundimage.tif");
backgroundTitle = getTitle();
open(input + File.separator + file);
fileTitle = getTitle();
imageCalculator("Subtract create", backgroundTitle, fileTitle);
...