Jupyter Notebook for ImageJ

Tags: #<Tag:0x00007fd542c19340> #<Tag:0x00007fd542c191d8> #<Tag:0x00007fd542c18fa8>


Hi folks,

I’m happy to present you the SciJava Jupyter Kernel project.

The idea of this project is to build a Jupyter kernel on top of the SciJava framework.

As you may know, Scijava is the Java framework that power ImageJ and Fiji.

Jupyter is an open source project that “support interactive data science and scientific computing across all programming languages”. Interactive computing is allowed by the Jupyter Notebook. Find here some notebooks that will give you an idea of what we are talking about.

The Jupyter SciJava Kernel try to make the link between both projects by providing a Java kernel powered by the SciJava Scripting libraries.


The easiest way to instal the kernel is to download the Python Anaconda distribution and install the SciJava kernel package from conda-forge. Here are the instructions.

A lot of effort is made to keep SciJava Jupyter Kernel independent of ImageJ/Fiji while keeping a smooth as possible out-of-the-box integration with it.

Scripting languages

Thank to the SciJava Scripting libraries, SciJava Jupyter Kernel support 8 languages by default :

  • Groovy
  • Python
  • BeanShell
  • Clojure
  • Java
  • JavaScript
  • Ruby
  • Scala


Here are some features of the kernel:

  • 8 languages supported.
  • Automatic standalone installation with conda.
  • Auto-completion support for all languages.
  • Automatic output conversion : Img and Dataset are converted on-the-fly and displayed within the notebook as PNG/JPEG. Same conversion is done for JSON, table, HTML, Markdown, etc.
  • Universal artifacts grabbing mechanism : use annotation parameters to grab Maven artifacts on-the-fly. Example: #@dependency(group="org.springframework", module="spring-orm", version="3.2.5.RELEASE").
  • Support many Jupyter clients: Jupyter Notebook, Jupyter Lab and Jupyter Console.

You can also browse the documentation notebooks.

Under The Hood

Under the hood, SciJava Jupyter Kernel uses the BeakerX base kernel. The BeakerX project is still young such as SciJava Jupyter Kernel and we are excited to grow together.

A big thanks to the BeakerX team for their project and the help.


SciJava Jupyter Kernel needs to be tested ! So feel free to try it an report any bug you may notice.

Bugs can be reported on Github or Gitter.


Comprehensive Fiji updates


Huge thanks to @hadim and @awalter17 for all their hard work on this exciting project! :beers:

In addition, I want to highlight that the ImageJ tutorial notebooks, previously developed in Beaker Notebook, have now been recast and updated to use the new SciJava Jupyter Kernel built on BeakerX.

If you are a software developer, bioimage analyst, data scientist, or just a fan of Jupyter, please check out the ImageJ tutorial notebooks for lots of goodies illustrating how to use the new ImageJ APIs, particularly ImageJ Ops and ImgLib2.


You can now install the kernel to any Python distribution with the SJK Fiji plugin. To install it you need to use the appropriate update site: https://github.com/scijava/scijava-jupyter-kernel#installation---with-fiji-integration


Thanks @hadim this looks really interesting.

I work a lot with jupyter notebooks in python using pandas, matplotlib and scikitlearn. I don’t know anything about Beaker notebooks other than I had thought they were good for combining multiple languages.

My own #1 use case for this, would be to do a bunch of work in scijava/imagej-ops, generate tables of image statistics then switch to python and analyze the data with pandas, and perform machine learning with scikitlearn.

I played around with the notebooks a bit, and couldn’t figure out if you can combine python toolkits and scijava. Is this possible yet??


Glad you like it.

I have the same workflow as you and use Jupyter/IPython a lot. There is no “easy” way, for now, to directly call SciJava stuff inside a CPython session (or the opposite).

Usually, I do my IJ images analysis with the Scijava kernel then export the results (CSV most of the time or TIF for images) and then load it with Pandas or tifffile with the CPython kernel.


That is certainly the dream target! But as @hadim said, it is not trivial to do yet. That said, here is a list of existing ways for accomplishing it now in some way, shape or form:

  1. The old standalone Beaker Notebook, still downloadable from http://beakernotebook.com/, does support transparent access of data between cells of different languages, via JSON marshaling internally. If you were to try passing huge amounts of data across a language boundary, you might notice, but for typical uses, it works well. There was an example notebook combining Python with pandas for data reading, plus Groovy for charting, on the Beaker Publications server, but it looks like that service has now been discontinued. My understanding is that the Two Sigma team is still very motivated to enable this sort of polyglot notebook, and that the BeakerX kernel will hopefully gain this feature over time.

  2. There is a SciJava script language called scripting-cpython, written by @LeeKamentsky, which offers a CPython script language. The syntax for passing objects across the JNI boundary is still not as elegant as it could be, but it is a good start at a feasible performant and general way of doing this sort of thing.

  3. Recent work by @hanslovsky lets you wrap numpy images as ImgLib2 data structures by reference! This is the imglib2-imglyb project. He also has a CPython REPL which works with ImageJ using this mechanism; see imagey.

  4. There is a RESTful image server backed by ImageJ which enables interprocess communication of ImageJ data structures between any language which speaks HTTP and JSON. We have proof of concept clients in Python (via a Python REPL) and JavaScript. This could be plugged in to a notebook to pass data between cells, among many other uses.

I do not have a clear vision of any “one true solution” along these lines, nor how we would/could/should combine the components above. I am hoping that others in the community have time to explore, evaluate and maybe even improve these solutions. It would be amazing if the people with these acute needs could contribute to the documentation and examples of how to achieve this sort of cross-language integration.


@bnorthan @ctrueden imglib2-imglyb (imglyb) runs on Linux and OSX, and so does imagey (just as of 5 minutes ago for OSX). If you run either of those two operating systems, you will be able to use it for your analysis. For windows, there is still an unsolved issue with PyJNIus. I do not know how important Windows support is for the developers of PyJNIus but if you run Windows and you are interested in imglyb it would help to raise their attention with a comment on the issue.

Conda packages exist for Linux and will be created for OSX, hopefully tonight:

conda install -c hanslovsky imglib2-imglyb

If you are interested in using imglyb you should check out imglyb-examples and the notebooks for the current Fiji learnathon. imglyb-examples is also available on conda:

conda install -c hanslovksy imglyb-examples

If you decide to use imglyb and have any questions, please do not hesitate to contact me. Also, the library is far from mature so now is the best time to raise any concerns or wishes. The best way to do so is to file an issue.


What about http://jyni.org? I know they are still very experimental, but that would avoid pushing large amounts of data to and from JVM.


I visited that web page about once a month for quite a few months, but never saw anything concrete enough to try to use. Recently, though, I saw a message on the Jython users mailing list suggesting it could be used in practice. And now, visiting the JyNI front page again, I see that as of a week ago, there is a functional alpha which works with numpy! Awesome! Have you tried it??


Not yet, but I do intend to!


I am also bugging @hanslovsky to try it with me while we are in Dresden together in person. If we manage to find time, we will report back here with our findings! :smile:


Say hello to Dresden from me :smiley:


@radoslaw.ejsmont @ctrueden That looks awesome! We could ship that with Fiji and use it to run imglyb, once it is stable and reliable enough. Do you know if Jython will implement Python 3.x anytime soon?


Unclear. It is being worked on, and you can see the current code.


We tried it!

TL:DR: It is not yet ready, but still a promising start. :wink:

$ java -cp jython.jar:build/JyNI.jar org.python.util.jython
Jython 2.7.1rc3 (default:562c2045cb60, Jun 17 2017, 16:27:47)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_131
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/usr/local/lib/python2.7/site-packages')
>>> import numpy
>>> image = numpy.zeros((2, 3))
>>> image.shape
(2, 3)

So far so good. Then:

>>> import skimage.data
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/skimage/__init__.py", line 158, in <module>
    from .util.dtype import *
  File "/usr/local/lib/python2.7/site-packages/skimage/util/__init__.py", line 5, in <module>
    from .apply_parallel import apply_parallel
  File "/usr/local/lib/python2.7/site-packages/skimage/util/apply_parallel.py", line 2, in <module>
    from multiprocessing import cpu_count
ImportError: No module named multiprocessing

So sad. No multiprocessing yet, so no scikit-image yet. :disappointed:

For people who care about this, the relevant issue is:


Even the support for numpy is still very alpha:

>>> import numpy; numpy.random.randint(5)
dyld: lazy symbol binding failed: Symbol not found: __PySequence_IterSearch
  Referenced from: /Users/curtis/code/tech/JyNI/build/libJyNI.so
  Expected in: flat namespace

dyld: Symbol not found: __PySequence_IterSearch
  Referenced from: /Users/curtis/code/tech/JyNI/build/libJyNI.so
  Expected in: flat namespace

./jy: line 3: 66829 Abort trap: 6           java -cp jython.jar:build/JyNI.jar org.python.util.jython $@

But nonetheless, Stefan Richthofer deserves mad props for even getting this far, and for continuing to push on it.


numpy.random.randint fails for me as well with a different failure mode:

>>> import numpy as np
>>> np.random.randint(3)
Traceback (most recent call last):
  File "mtrand.pyx", line 970, in mtrand.RandomState.randint (numpy/random/mtrand/mtrand.c:15903)
TypeError: argument of type 'dict' is not iterable


Impressive! Future looks promising :slight_smile:


Breaking news: @hanslovsky just confirmed that with one minor tweak to the configuration, he was able to create a numpy array, convert it to an ImgLib2 data object using his ImgLyb library, and then show it in BDV, all from Jython:

import imglyb
import imglyb.util as util
import numpy as np

arr = np.random.rand( 300, 400 ) * 2**16
rai = imglyb.to_imglib( arr )
util.BdvFunctions.show( rai, 'rai', util.BdvOptions.options().is2D() )

More details later. :sunny:


Is matplotlib working?