ImageJ-MATLAB: IJM.show() does not reliably pass numeric data to ImageJ?

matlab
32bit
imagej-matlab
Tags: #<Tag:0x00007fa30b31ff00> #<Tag:0x00007fa30b31fd98> #<Tag:0x00007fa30b31fc30>

#1

I’ve encountered a very strange bug with regard to pixel values of image data passed to ImageJ/Fiji from MATLAB (R2018a) via ImageJ-MATLAB (version 0.7.2).

At the moment, although I’m not sure this is directly connected to my main issue, I found a possible bug in IJM.show()

In the example below, I opened a demo RGB image data (A) in MATLAB, sent the data to ImageJ with IJM.show('name') and then retrieved the integer values of image data by directly using ImageJ Java API (B as int32 and C as uint8) and IJM.getDatasetAs('name') (D as uin8 converted from double).

Although the values of C and D were identical, indicating that IJM.getDatasetAs() is working properly, those of A and C were slightly different. As in the attached zip file, results of imshow() commands look almost identical, so the differences were quite subtle. In fact, the maximum difference between arrays A and C was only 5 (A being larger than C by 0 to 5 pixel values).

As to B_, which is in int32 data type, the integer values are much larger than the range of uint8, hence B is almost saturated. I suspect that the problem lies at this 32bit data type or conversion to 8bit data type.

By any chance, if there is away to retrieve the correct 8bit numeric data from the 32bit data, please let me know.

In the code below, as far as I’m aware I only used builtin MATLAB functions and ImageJ Java API, so you should be able to reproduce it.

I have tested a similar code for a single channel image, and the numeric values were kept identical between MATLAB and ImageJ.

Because this does not issue an error, and the image looks almost as good as it should be, it was hard to pin this down.

If this is a bug, please can somebody fix this!? Otherwise, I might end up wasting a whole month of work…

clear -regexp ^((?!IJM).)*$;close all;clc

if ispc
    addpath 'D:\Fiji.app\scripts' % needs tomodify this according to Fiji installation
else
    addpath '/Applications/Fiji.app/scripts'
end
ImageJ

[A] = imread('peppers.png');

imshow(A)

A_ = permute(A,[2 1 3]);

IJM.show('A_')

imp = ij.IJ.getImage(); % automatically converted to 32 bit per channel

RGB channels are mistakebly interpretted as 3 slices in a stack of single channel image

d = imp.getDimensions;
B_ = zeros(d(1),d(2),d(3),'uint32');
for s = 1:3
    imp.setSlice(s)
    ip = imp.getProcessor();
    B_(:,:,s) = ip.getIntArray();
end

B_ = permute(B_,[2 1 3]);

B = uint8(B_);

imshow(B)

Bcause IJM.show() opens an image in 32bit format, you need to convert itn back to 8 bit

ij.IJ.run(imp, "8-bit", "");

imp2 = ij.IJ.getImage(); % automatically converted to 32 bit per channel
d = imp2.getDimensions;
C_ = zeros(d(1),d(2),d(3),'uint8');
for s = 1:3
    imp2.setSlice(s)
    ip2 = imp2.getProcessor();
    C_(:,:,s) = ip2.getIntArray();
end

C = permute(C_,[2 1 3]);

imshow(C)

Retrieve numeric data with IJM.getDatasetAs()

IJM.getDatasetAs('D_')

D_ = permute(D_,[2 1 3]);

class(D_)

D = uint8(D_);

imshow(D)

Validation of results

isequal(A,B)
ans = 
   0
isequal(B,C)
ans = 
   0
isequal(C,D)
ans = 
   1
A(1:3,1:10,1)
ans = 3×10 uint8 matrix    
   62   63   63   65   66   63   61   63   63   64
   63   61   59   64   63   60   61   64   64   63
   65   63   63   66   66   62   60   66   64   64
B_(1:3,1:10,1)
ans = 3×10 uint32 matrix    
   1115160576   1115422720   1115422720   1115815936   1115947008   1115422720   1114898432   1115422720   1115422720   1115684864
   1115422720   1114898432   1114374144   1115684864   1115422720   1114636288   1114898432   1115684864   1115684864   1115422720
   1115815936   1115422720   1115422720   1115947008   1115947008   1115160576   1114636288   1115947008   1115684864   1115684864
B(1:3,1:10,1)
ans = 3×10 uint8 matrix    
   255   255   255   255   255   255   255   255   255   255
   255   255   255   255   255   255   255   255   255   255
   255   255   255   255   255   255   255   255   255   255
C(1:3,1:10,1)
ans = 3×10 uint8 matrix    
   58   59   59   61   62   59   57   59   59   60
   59   57   55   60   59   56   57   60   60   59
   61   59   59   62   62   58   56   62   60   60
D(1:3,1:10,1)
ans = 3×10 uint8 matrix    
   58   59   59   61   62   59   57   59   59   60
   59   57   55   60   59   56   57   60   60   59
   61   59   59   62   62   58   56   62   60   60

HTML and MATLAB .mlx files

scr20180502_065604_ImageJMatlab_RGB_issue.zip (2.3 MB)


#2

#3

I’ve further examined with a single channel image and it turned out that my use of ij.process.ImageProcessor.getIntArray(), which returns huge values, is probably wrong, and I should use ij.process.ImageProcessor.getFloatArray() to retrieve integer values from ImageJ in single data type. When accessed with this method, the 32bit image opened in ImageJ still holds the right pixel values. So, for single channel image everything is fine now.

scr20180502_083216_ImageJMATLAB_singlechannel.zip (2.0 MB)

Back to 3 channel images, further tests with ij.process.ImageProcessor.getFloatArray() showed that IJM.show() maintains the correct pixel values.

It appears that the image type conversion by ij.IJ.run(imp, "8-bit", "") from 32bit to 8bit is actually causing the problem.


#4

Yes, I suggest to be careful when doing conversions in ImageJ 1.x (i.e. using the ij.IJ class), as it depends on the current state of your ImageJ (in case of RGB to 8-bit conversion, the Weighted RGB conversions option) and the current image display (in case of 32-bit to 8-bit).

From the user guide:

8-bit Converts to 8-bit grayscale. ImageJ converts 16-bit and 32-bit images to 8-bit by linearly scaling from min–max to 0–255, where min and max are the two values displayed in the Image▷Adjust▷Brightness/Contrast… [C]. Image▷Show Info… [i] displays these two values as Display range. Note that this scaling is not done if Scale When Converting is not checked in Edit▷Options▷Conversions…. RGB images are converted to grayscale using the formula gray = (red + green + blue) ⁄ 3 or gray = 0.299 × red + 0.587 × green + 0.114 × blue if Weighted RGB Conversions is checked in Edit▷Options▷Conversions….


An RGB image is a 3-channel, 8-bit image, so that’s how it should be treated, no? Maybe the metadata got lost that the slices represent channels in your case.


#5

Thanks a lot.

I just figured it out. A multichannel image opened with IJM.show('foo') is in the size of [j,i,1,1,k] where i, j and k being the size of X and Y dimensions and the number of channels of foo, respectively. And by default ImageJ interpret this data in XYCZT format. So, the X and Y axes are flipped over, while the channels are treated as time frames rather than Z slices or channels!

ij.IJ.run("Re-order Hyperstack ...", "channels=[Frames (t)] slices=[Slices (z)] frames=[Channels (c)]");

can fix this and you’ll get a single plane multi-channel images.

As to the bit depth conversion, I’m getting fed up with this! convertToByte(0) method of ImageProcessor class appears to be the one to be used (option 0 refuses scaling), but I can’t figure out how to treat the multiple channels. So far when I try to use this method for one channel, I end up replacing the other 2 channels with the first one.


#6

Is there any chance you can avoid using ImageJ 1.x structures such as ij.ImagePlus, and instead use net.imagej.Dataset? I fear that this adds to the confusion, because you have to deal with the Matlab<>IJ2 conversion (done internally by the ImageJ-MATLAB component) at the same time as with the IJ1<>IJ2 conversion (by the ImageJ legacy layer in Fiji), and issues might arise on either of those.


#7

I’ve figured it out.

There might be other ways to do it, but as I expected, ImageProcessor.convertToByte(0) with 0 input is the key.

For the conversion of image data type, I collected ImageProcessor objects for each channels. And then prepared an ImageProcessor object as a 5D HyperStack and then assigned the byte processors (8 bit) for each channel.

Then, I was able to maintain the integer values throughout the process!

[A] = imread('peppers.png');

imshow(A)

A_ = permute(A,[2 1 3]);

IJM.show('A_')

imp = ij.IJ.getImage(); % automatically converted to int32 (not a hyperstack)
d = imp.getDimensions();
st = imp.getStack();

ip = cell(d(5),1);
for s = 1:d(5)
    ip{s} = st.getProcessor(s);
end

% create new ImagePlus wiht 8bit data
imp2 = ij.IJ.createHyperStack('new', d(1),d(2),d(5), 1, 1, 8);

for ch = 1:d(5)
    imp2.setC(ch);
    imp2.setProcessor(ip{ch}.convertToByte(0)); % 0 defies scaling, use convertToShort() for 16bit data
end

imp2.show();
imp2.setDisplayMode(ij.IJ.COLOR) %NOTE this is required to enable the next line
imp2.setDisplayMode(ij.IJ.COMPOSITE)
imp2.updateAndDraw()


E_ = zeros(d(1),d(2),d(3),'single');
for s = 1:3
    imp2.setC(s)
    ip2 = imp2.getProcessor();
    E_(:,:,s) = ip2.getFloatArray();
end

E_ = permute(E_,[2 1 3]);

E = uint8(E_);

imshow(E)

IJM.getDatasetAs('F_')

F_ = permute(F_,[2 1 3]);

class(F_)

F = uint8(F_);

imshow(F)

isequal(A,E) % equal!!!

isequal(A,F) % equal!!!

#8

In order to solve this annoying issues related to IJM.show I wrote a MATLAB function ijmshow that works as a wrapper of IJM.show(). Now it can handle 5D stacks properly with numeric values mtaintained for 16bit or 8bit images.


#9

Improved one.

Pull Request was created.