Studying plugins: Converting to ArrayImg, and OpService.math() usage questions

imglib2
plugins
imagej-ops
Tags: #<Tag:0x00007fd69cbaa908> #<Tag:0x00007fd69cbaa750> #<Tag:0x00007fd69cbaa610>

#1

I have set myself the task of writing a simple IJ2-style
plugin that takes an open image, blends it with an image
retrieved from a resource in the plugin’s jar, and stores
the result in the open image.

I will start with a specific question, and then ask about
more general OpService usage.

To start, I am trying to get OpService.math().multiply()
to work. My problem is that it seems to require an
ArrayImg, while I have an Image (or ImagePluse or Img).
I also don’t see any prepackaged method for converting
various flavors of images to an ArrayImg. (I see the
same issue for math().divide().)

Specifically:

   // from the internet:
   // IJ1 (?) method for opening an image stored as a resource
   url = getClass().getResource (resourcePath + imgName);
   Image image = Toolkit.getDefaultToolkit().getImage(url);            
   opService.math().multiply (image, 2.0);

gives the compile-time error:

   MyPlugin.java:94: error: no suitable method found for multiply(Image,double)
      opService.math().multiply (image, 2.0);
                      ^
    method MathNamespace.<B#1>multiply(ArrayImg<B#1,ByteArray>,byte) is not applicable
      (cannot infer type-variable(s) B#1
        (argument mismatch; Image cannot be converted to ArrayImg<B#1,ByteArray>))
    method MathNamespace.multiply(ArrayImg<DoubleType,DoubleArray>,double) is not applicable
      (argument mismatch; Image cannot be converted to ArrayImg<DoubleType,DoubleArray>)
    method MathNamespace.multiply(ArrayImg<FloatType,FloatArray>,float) is not applicable
      (argument mismatch; Image cannot be converted to ArrayImg<FloatType,FloatArray>)
    ...
    (more of the same)

I see similar errors if I wrap the Image in an ImagePlus,
and if I use an Img obtained from the input harvester via:

   // from FrangiVesselness.java
   @Parameter
   private Img<T> input;

First question:

How do I (assuming that using OpService.math() is the
appropriate IJ2 way to blend two images together) turn the
various image beasties into an ArrayImg? Or can I get
OpService.math() to operate on other kinds of images?

Second question:

Is my plan of using OpService.math() (basically multiply
and add) to blend two images together a sensible “IJ2
way” of proceeding? Would you have any pointers in advance
for me to help me along my way?

Third (minor) question:

Is there a better, IJ2-approved way to fetch an image from
a resource in the jar?

Again, my (educational) specification is to take an existing
image that is open in Fiji, and blend it in place with a
resource image.

Thanks for any advice, mm.


#2

Hi @mountain_man

Not sure if this helps, but here is a simple script that multiplies an image using ops, 2 ways… it seems to work for me.

The second way uses the actual Type of the image. It is more complicated, but it may help, if for some reason, the op matcher isn’t working for the first approach. It could be that the combination of Img class (ie ArrayImg, PlanarImg etc), and data Type (ie ShortType, FloatType, etc) could be confusing the matcher somehow, if so let us know exactly what class and type your image is.

// @ImageJ ij
// @OUTPUT ImgPlus(label="multiplied1") multiplied1
// @OUTPUT ImgPlus(label="multiplied2") multiplied2

resource="http://imagej.net/images/fluoview-multi.tif"
image = ij.io().open(resource);  // convenient example stack
image=image.getImgPlus().getImg();
ij.ui().show(image)

// approach 1, simply multiply by 2
multiplied1= ij.op().math().multiply(image, 2.0);

// approach 2, get element from image, set value, and multiply
temp=image.firstElement().copy();
temp.setReal(2.0);
multiplied2= ij.op().math().multiply(image, temp); 

#3

Hello Brian -

Thanks for your reply.

I’m having trouble making progress with your suggestions, probably
because of complications translating from the IJ macro language
to java.

Anyway, when I try your second approach – passing multiply the
temp variable to let it know the specific type – I get stuck on
getFirstElement:

MyPlugin.java:80: error: cannot find symbol
Object temp = imageB.firstElement().copy();
^
symbol: method firstElement()
location: variable imageB of type ImagePlus

Basically, in:

  Image image = Toolkit.getDefaultToolkit().getImage(url);            
  ImagePlus imageB = new ImagePlus("imageB", image);
  Image imageC = imageB.getImage();

image is of actual type sun.awt.image.ToolkitImage, imageB is an
ij.ImagePlus, and imageC is a java.awt.image.BufferedImage.
None of these classes has a getFirstElement method (or similar),
so I’m at a little bit of a loss.

I’m starting to wonder if I’m barking up the wrong tree here, trying
to use OpService.math() with images. Even if this approach can
be made to work, the contortions it seems I have to go through
suggest that there might be a better way.

It seems to me that a natural approach might be to convert an
"image" (ImagePlus, etc.) to an ArrayImage, but i wasn’t able
to find any relevant constructors or conversion functions. I ought
to be able to write one myself, but that would seem to be against
the spirit of the OpService framework.

Any further suggestions would be appreciated.

Thanks, mm.


#4

Here is the java code. Interestingly the first approach doesn’t compile in java presumably because there is no signature that takes a generic type Img<T> and a number. So you have to use the signature that uses a Type.

import java.io.IOException;

import net.imagej.ImageJ;
import net.imglib2.img.Img;
import net.imglib2.type.NativeType;
import net.imglib2.type.numeric.RealType;

public class MultiplyTest {

	final static String inputName = "./C2-confocal-stack.tif";

	final static ImageJ ij = new ImageJ();

	public static <T extends RealType<T> & NativeType<T>> void main(final String[] args)
			throws InterruptedException, IOException {

		ij.launch(args);

		Img<T> image = (Img<T>) ij.io().open("http://imagej.net/images/fluoview-multi.tif"); // convenient
																								// example
																								// stack
		
		ij.ui().show("original",image);
		
		// approach 1, simply multiply by 2
		// DOESN'T COMPILE BECAUSE WE DON'T KNOW Type???????
		// Img<T> multiplied1= ij.op().math().multiply(image, 2.0);

		// approach 2, get element from image, set value, and multiply
		T temp = image.firstElement().copy();
		temp.setReal(2.0);
		Img<T> multiplied2 = (Img<T>) ij.op().math().multiply(image, temp);

		ij.ui().show("multipled",multiplied2);
	}

}

#5

Hi Brian -

Thank you for your follow-up – I am making (partial) process.

Yup. Doesn’t work for me either without passing the second argument
as a specifically-typed variable.

Following your sample code, I can get part of my (educational) task
done. I can use OpService.math to multiply an open image that I
access with @Parameter in my plugin. (It doesn’t work all the way,
for reasons I will ask about in a separate post.)

Roughly:

  @Parameter
  private Img<T> input;

  T temp = input.firstElement().copy();
  temp.setReal (2.0);
  Img<T> input2 = (Img<T>) opService.math().multiply (input, temp);
  ImageJFunctions.show (input2, "input2");

This part – multiplying a @Parameter open image – works*. (*almost)

I still can’t OpService.math().multiply an image that I read from an
in-jar resource. The sticking point remains that I can’t convert (or
wrap) my Image / ImagePlus to an Img:

Browsing some internet examples and the javadoc, I came across
two “wrap” functions that seem to be suggested as the way to
use imglib2 with ImageJ1 images. But I can’t get either to work:

  url = getClass().getResource (resourcePath + imgName);
  Image myImage = Toolkit.getDefaultToolkit().getImage(url);            
  ImagePlus myImagePlus = new ImagePlus("myImagePlus", myImage);
  myImagePlus.show();

  // these two wrap attempts fail at compile time ...
  // Img<T> myImagePlusWrap = ImagePlusAdapter.wrap (myImagePlus);
  // Img<T> myImagePlusWrap = ImageJFunctions.wrap (myImagePlus);

Both versions of wrap() give essentially the same compile-time error:

MyPlugin.java:83: error: incompatible types: inference variable T#1 has incompatible bounds
      Img<T> myImagePlusWrap = ImagePlusAdapter.wrap (myImagePlus);
                                                     ^
    equality constraints: T#2
    upper bounds: Object,NumericType<T#1>,NativeType<T#1>
  where T#1,T#2 are type-variables:
    T#1 extends Object,NumericType<T#1>,NativeType<T#1> declared in method <T#1>wrap(ImagePlus)
    T#2 extends RealType<T#2> declared in class MyPlugin

...

MyPlugin.java:84: error: incompatible types: inference variable T#1 has incompatible bounds
      Img<T> myImagePlusWrap = ImageJFunctions.wrap (myImagePlus);
                                                    ^
    equality constraints: T#2
    upper bounds: Object,NumericType<T#1>,NativeType<T#1>
  where T#1,T#2 are type-variables:
    T#1 extends Object,NumericType<T#1>,NativeType<T#1> declared in method <T#1>wrap(ImagePlus)
    T#2 extends RealType<T#2> declared in class MyPlugin

myImage / myImagePlus is read from a .jpg image file stored as a
resource in the plugin’s jar. When displayed by Fiji, it advertises
itself as "512x512 pixels; 8-bit; 256K. The specific type of myImage
is sun.awt.image.ToolkitImage, and the specific type of myImagePlus
is ij.ImagePlus.

It looks like I am still struggling with the interface between ImageJ1
and ImageJ2. Remember, my (educational) use case is to write an
IJ2-style plugin to use with a stock Fiji install, and it looks like Fiji
still lives mostly in IJ1 land, so (based on my chosen use case), I
do have to deal with the IJ1 / IJ2 divide.

Any further wisdom would be greatly appreciated.

Thanks, mm.


#6

Following up on my own post, let me summarize the solutions to
my two main sticking points:

First, wrapping an ImagePlus into an Img<T>:

My error here is not casting the result of ImageJFunctions.wrap().
The correct code is:

  Img<T> myImagePlusWrap = (Img<T>) ImageJFunctions.wrap (myImagePlus);

This now works, and I can go on and work with myImagePlusWrap,
for example, by multiplying it using OpService.math().multiply().

I don’t really understand why the explicit (Img<T>) cast is
necessary; I guess it has something to do with the java generic
type variable T.

For completeness, let me repeat the solution to getting
math().multiply() to work. Following Brian’s explanation,
multiply() needs to be passed an explicitly-typed variable
(variable temp, below) for the correct function signature to
be resolved:

  @Parameter
  private Img<T> input;

  T temp = input.firstElement().copy();
  temp.setReal (2.0);
  Img<T> input2 = (Img<T>) opService.math().multiply (input, temp);
  ImageJFunctions.show (input2, "input2");

Thanks for your help, Brian, mm.