Orthogonal views are slow when showing imglib2 images in legacy UI

imglib2
imagej-legacy
Tags: #<Tag:0x00007fa30768f360> #<Tag:0x00007fa30768f180>

#1

@tpietzsch @maarzt

Using the legacy UI for showing imglib2 images is slow when using the “Orthogonal Views” functionality of ImageJ1. Currently there are no concrete plans for how to improve this. The function of this post is to serve as a discussion forum for this issue.


#2

This can be reproduced in an up to date FIJI:

  • Opening an image with about 600x600x600 pixels as cell image. (File > Import > Image…)
  • Run Image > Stacks > Orthogonal Views

It takes several seconds to show the orthogonal views. And they respond only very slowly.


#3

There is a simple workaround:

Duplicate the image stack before applying Orthogonal Views.


#4

Orthogonal Views are slow on virtual stack.

Why is it slow:

  • It’s cause by the IJ1 ImagePlus memory layout. Images are stored in memory as planar images. To show an orthogonal slice of a planar image, every plane (the entire image) must be loaded into memory. This is no problem with regular ImagePlus as the planar image is kept in the memory all the time. But it’s slow for virtual stacks.

Possible solutions:

  1. The workaround :wink: described earlier

  2. Use imglib2 planar images.

  3. Add a cache to the virtual stacks used for displaying Imglib2 images. This would basically automate the step of duplicating the image for orthogonal views. And therefor shares the disadvantages:

    • Only works for images that fit into memory
    • The view isn’t updated when the image changes

    But the main disadvantage is the increased complexity: The cache needs to be refreshed when the image content changes. For large images the cache could quickly fill the complete memory, causing more complex garbage collection.

  4. Implement a new IJ2 orthogonal views plugin. Maybe based on big data viewer.


#5

My vote goes for this one :slight_smile:


#6

That would be very useful to have indeed.

Maybe the OrthogonalViews class from saalfeldlab/paintera could somehow be migrated upstream into the bigdataviewer organisation and made available in Fiji?

@hanslovsky, @axtimwalde, @tpietzsch what do you think?


#7

We could use OrthogonalViews to implement a new IJ2 orthogonal views plugin. If people are interested in that kind of solution, I would create a basic example that could be extended in a commnuity effort. Please point me to the code of the existiing Original Views plugin.

The most difficult part will probably be the integration of OrthogonalViews (JavaFX) into an awt/swing UI. This is also the reason, why the Paintera OrthogonalViews class should not be part of bdv but there could be a separate project somewhere in the SciJava space for these kind of UI elements.


#8

The source of the ImageJ1 plugin Orthogonal Views is here:


#9

FYI, I started to create a minmal example and I can show data (without controls yet), and it worked. Next step:

  • decide which parts of paintera need to be separated out and create repo for that (my task)
  • replicate behavior of old ortho views (if desired, someone else)

#10

Unfortunately, wrapping my OrthogonalViews class in a JFXPanel is prohibitively slow (I assume the problem is mixing awt/swing with JavaFX). I haven’t found a way yet to start the JavaFX application thread without JFXPanel