BigDataViewer for interactive registration

Tags: #<Tag:0x00007fd69be6a4c8> #<Tag:0x00007fd69be6a388> #<Tag:0x00007fd69be6a248>



I am trying to implement an interactive registration by building on the BigDataViewer.

The functionality:

  1. load an image volume (3D) in BDV
  2. load an additional image section (2D section-image) in BDV
  3. keep the section-image fixed, while moving throught the image volume.
  4. align the currently visible section of the image volume to the section-image by triggering with a keyboard shortcut or maybe an with a button on an additional dialog
  5. get the final section of the image volume and it’s transform on the section-image

Additionally, I though it might be useful to create an additonal panel, showing a projection of the volume with the current section/plane to help the user navigate.

To keep the section image fix, I added a dimension to it using.

RandomAccessibleInterval rai = ...
RandomAccessibleInterval secImg = Views.addDimension(rai, 0, zmax)

This replicates the section along the 3rd dimension to match the image volume. While it does the trick for one given 2D perspective, it looks confusing when switching the rotation axis. Ideally I would be able to fix the roation axis. Is there a way to do that?

Registering an additonal key-event was easy with BdvHandlePanel().getKeybindings().addInputMap() and the corresponding action with BdvHandlePanel().getKeybindings().addActionMap()

But even after reading throug BDV publication, I could need some pointers to navigate the BDV code to find answers for:

  • How to extract the currently visible section of the volume as RandomAccessibleInterval?
  • How to get the current plane in the global coordinate system?
  • How to apply a transform to the image volume?


after some digging around I found that the transforms are accessible with:

ViewerPanel bdvPanel = ...
ViewerState state  = bdvPanel.getState();

final AffineTransform3D T_vw = new AffineTransform3D();

SourceState srcState = state.getSources().get(sourceNumber)
final AffineTransform3D T_wl = new AffineTransform3D();
srcState.getSpimSource().getSourceTransform(0, 0, T_wl);

final InvertibleRealTransformSequence T_vl = new InvertibleRealTransformSequence();

And the I can get the RandomAccessibleIntervals with:

RandomAccessibleInterval rai = srcState.getSpimSource().getSource(0, 0);

I did not see any methods to directly change the transforms on the sources. But I saw, that I can remove and add them.


So far so good…
The remaining question is if the removing and adding the sources is the proper (most efficient) way to do it. Is therea better way, where I just can add a transform on a given source, that will then be interpreted upon rendering?


I’m not sure, but did you have a look at the BigWarp plugin? It seems to use bdv.img.WarpedSource for its viewer panel sources.

Maybe @bogovicj has more advice…


no, I had a look at BigStitcher, was almost taking the two for the same :confused:. Thanks for the hint.


Ya, somehow it’s all Big* these days… :wink:


Hi @FelixM,

The most helpful thing (I think) for you will be this:
Bdv ViewerPanels have a State that tell you what the “viewer” or “camera” transformation is - and from this you can figure out what slice of the 3d volume you’re looking at.

Something like

state = bdv.getViewer().getViewerState();
// copy the transform into the my Transform variable
myTransform = new AffineTransform3D();
state.getViewerTransform( myTransform ); 

I seem to remember in a different forum post, you asked about something similar and got the ‘Manual Transform’ feature of bigdataviewer - is that right? Have you been able to transform volumes with this?

If a translation/scaling transformation is sufficient for this task, then BigWarp may be overkill - its main purpose is to do interactive nonlinear (thin plate spline) transformations.

Sort of … there’s a way to specify that you’re working in 2D, and this fixes the rotation axes, though I’m not sure whether this will also break the interaction you want to have with the 3D source that’s loaded … I’ll look around for this and post back.


Edit: typo fix


Follow up on the “fixing the rotation axis”

I think this code will do the trick (but havn’t tested).

// if you're using the BigDataViewer class 
   setups, sources, numTimepoints, cache, title, progress,
   ViewerOptions.transformEventHandlerFactory( BehaviourTransformEventHandlerPlanar.factory() )

// if you're using the BdvFunctions helpers img, name, BdvOptions.is2D() );

for example.


Thanks for your input @bogovicj

yes, I was able to use this.
To recapitulate:

BdvHandlePanel bdvHandle = ...

// From source into the global space
SourceState srcState = viewerState.getSources().get(0);
final AffineTransform3D T_wl = new AffineTransform3D();
srcState.getSpimSource().getSourceTransform(0, 0, T_wl);

// From the global state to the viewer (who is showing plane z=0)
ViewerState viewerState = bdvHandle.getViewerPanel().getState()
AffineTransform3D T_vw = new AffineTransform3D();

The manual transform becomes the source transform when applied, right?

To change a tranform I would first get it via BdvHandlePanel and then change it. To apply it I would remove BdvSource().removeFromBdv() and then add it again with bdvHandle.
I do not know if this is the way BDV-authors envisioned the implementation of this funcitonality, or if there is a more elegant way.
First I was looking for a setSourceTransform() method somewhere, but could not find one so far.

BdvOptions.options().sourceTransform() does not take InvertibleRealTransformSequence as input. So in this case I just worked directly on the source image and added it again.

Is there a more elegant way to deal with transforms and sources?
I would be happy to learn about that.

I played around with 2D images, but its making it more difficult to handle (to find the section and not scroll through it). So for now stick with the added dimension to the section.

Another quesiton while I am at it:
Why is the ViewerPanel().align() method is protected?
I would have liked to call it to change between XY, ZY and XZ planes.


It is concatenated to the source transform, see here, so if you want only the “manually changed” part, there will probably be an extra step.

Not more elegant, but more general perhaps, I created a WarpedSource class for bigwarp that is similar to Bdvs TransformedSource but uses an InverseRealTransform.

Not sure, sorry. :man_shrugging:


Yes, @imagejan told me to look at this. So you would just create a BvdSource and throw it in the Constructor of WarpedSource?


well, since Bigwarp still uses the more verbose bigdataviewer-core interfaces, it’d have to be something like wrapping every element of bdvStackSource.getSources()

I should update this at some point…