Discrepancy between different ways of displaying Views

imglib2
scijava
Tags: #<Tag:0x00007fb8849b8568> #<Tag:0x00007fb884a47060>

#1

Getting more and more nervous why imglib2.view did not produce the results I expected it to, I finally realized that there is a difference how images are displayed; imglib2 vs. scijava.display. So imglib2 seems to work fine but…

The original:

Applying a Views.interval() to extract a roi in the center and displayed with ImageJFunctions.show():

Applying a Views.interval() to extract a roi in the center and displayed with scijava.display:

So the size of the interval is correct, but the offset is not taken into account.

Here is the code to reproduce (with pom-scijava 14.0.0)

@Plugin(type = Command.class)
public class Sandbox implements Command {

    @Parameter
    OpService opService;

    @Parameter
    DisplayService displayService;

    @Override
    public void run() {

        int[] center = new int[] {15,10};
        Dimensions dims = new FinalDimensions(31, 21);

        Img<BitType> img = opService.create().img(dims, new BitType());

        RandomAccess<BitType> ra = img.randomAccess();
        ra.setPosition(center);
        ra.get().set(true);

        displayService.createDisplay("original", img);

        IntervalView view = Views.interval(img, new long[]{5, 5}, new long[]{25, 15});
        ImageJFunctions.show(view, "view - ImageJFunctions.show()");

        displayService.createDisplay("view - DisplayService.createDisplay()", view);
    }

    public static void main(String[] args) throws IOException, TransformerException, URISyntaxException {
        ImageJ ij = net.imagej.Main.launch(args);
        ij.command().run(Sandbox.class, false);
    }
}

Looks buggy to me…


Separating channels via Views.interval not working as expected
#2

Sorry, non-zero interval offsets are not well tested in the imagej-common and imagej-ui-swing display layers. They were not part of the original use case when developing those parts of ImageJ2 long ago, and they predate ImgLib2’s Views class.

It is likely straightforward to fix the relevant code, but I personally have too many other high-priority projects going on to investigate right now. But PRs welcome if you have time to dig a little. Let me know if you need pointers.


#3

Hi @ctrueden
thanks for the feedback.
I won’t make any promises here, but you can start by indicating the correct repository, then I’ll start by filing the bug report.


#4

I do not know which repository has the bug. There is an interplay between three different components here:

Every DatasetView can express itself as an ARGBScreenImage:

And the Swing UI uses that to paint the image into a JHotDraw figure:

That ARGBScreenImage is populated by a CompositeXYProjector as follows:

But nothing in the above code looks super suspicious to me (e.g., a loop from [0, w) and [0, h) or some such, ignoring interval minimums). The projector does take care to respect the interval minimums. So… further digging needed.


#5

Hi @ctrueden

I tried to deubg this issue.
The obvious workaround is to wrap the IntervalView before passing it to the DisplayService:

ImagePlus imp = ImageJFunctions.wrap(view, "wrapped");
displayService.createDisplay("this is rendered correctly", imp);

Comparing the the Display ouputs from the DisplayService ended up where you indicated; in the map() method of the CompositeXYProjector.
To me the following line looks suspicious:


In the method the target bounds are set without offset with the correct resulting size, but when defining the access on the source, shouldn’t the bounds be taken from the source too?


#6

No, this is correct. The area of source (which could be potentially much larger) corresponding to the target is required for the RandomAccess


#7

@tpietzsch in this particular case (see images above) min and max taken from the target do not take into account the offset from source. Let’s say source has a size of 35x25 and the interval of the view is defined from 5 to 30 in x and 5 to 15 for y, then the target RandomAccessible has the dimensions 25x10, but its minimum will be set to 0,0 and thus it won’t take the pixels defined by the interval of the view, but a View without the offset.


#8

@FelixM min and max shouldn’t take into account the source offset.
The point of view of the projector is:
There is a (2D, bounded) RandomAccessibleInterval target surface that must be filled with values
There is a RandomAccessible source from which the values can be obtained. The source is for this purpose infinitely large.
Any transformations that need to be done should happen outside of the projector. Even if the source has interval boundaries, there is no reasonable default behaviour.

For ImageJFunctions.show(...) (and ImageJFunctions.wrap(...)) the transformation happens here:

So the correct workaround would be passing Views.zeroMin(view) to the ImageJ2 display.

@ctrueden Possibly it would make sense to do this automatically in the display.
The alternative approach would be to take the projector approach and say the display has an interval but the source does not. On one hand, this would make it harder for first-time users. On the other hand, it makes more sense if you want to put multiple things into one display that have different intervals.


#9

Thank you @tpietzsch for elaborating on the subject. I think I understand now, that I was looking for the error in the wrong layer.

Given that you discribe the solution above to be a workaround and the discussion so far I am usure though what the expected behaviour actually is of calling the DisplayService on a IntervalView?


#10

I think it should behave the same as the ImageJFunctions.show: the interval bounds should be shifted such that the projector displays the entire bounding box of the interval.

I am spending a few minutes now looking at where to put the Views.zeroMin call. Your original MCVE above is very helpful for testing this—thank you!


Puzzling behaviour in image display
#11

I took a really quick stab at fixing this in imagej-common; see the fix-interval-offset branch. Unfortunately, no dice: there is no change in behavior in the MCVE. Time to spin up the debugger, I suppose. I am out of time for today, though—maybe later this week or next.