Using ImageJ with Android

android
imagej
Tags: #<Tag:0x00007fd540b273f8> #<Tag:0x00007fd540b272b8>

#1

I want to use ImagePlus and ImageProcessor in my Android application for some processing on captured images but ImageJ is dependent on Java AWT which is not supported by Android. Resources state that ImageJ2 has no dependency on AWT but is it complete and does it have the same functionalities (classes and methods) provided by ImageJ? (I need to use ByteProcessor, FloatProcessor methods like isBinary(), resetRoi() and many more). Or else is there any equivalent library in Android which serves the same purpose as ImagePlus and ImageProcessor? Any one who tried this before? Please help!


ImageJ2 for Android application
ImageJ would be great on touch screen
#2

The ImageJ2 API is a total rewrite, with no dependency on ImageJ 1.x. The ij package and subpackages are not provided; instead, classes of ImageJ fall under the net.imagej package hierarchy, as well as net.imglib2 and org.scijava for the foundational ImgLib2 and SciJava layers, respectively.

Great care was taken to avoid use of AWT in all these foundational layers. That said, there may be uses of other Java APIs not present on Android; for example, SciJava Common uses javax.script and javax.xml. I am not certain exactly which packages we use which are unsupported on Android. The best way to find out would be to try it.

Start by taking a look at the ImgLib2 library and its tutorials:

Note that some examples convert ImgLib2 data structures into ImageJ 1.x structures for display purposes; obviously you would not be able to use those functions (all part of the imglib2-ij component) in your Android app.

Unfortunately, no one is (yet) using ImgLib2 or ImageJ2 on Android in anger. So if you try it, expect numerous obstacles. However, it would be very exciting to get this working, and is certainly something I have wanted to do for several years. If you have the time and energy to explore it, I am happy to help in whatever ways I can. (E.g.: ultimately I would like to have a full Android build on our CI system to ensure that the APIs work on Android.)

Lastly, I want to mention that we have a complementary plan to getting ImageJ2 working on Android: a RESTful image server. Then, with some javascript front-end work, ImageJ2 would be useable on mobile as a client to a server workstation running the ImageJ server.


#3

Thank you for your reply. That helps! I will definitely try to follow your instructions. I will let you know if I face any issues! Thank you for all your help and support.


#4

Has anyone tried to use ImageJ2 in an Android App since the last post?

If not, I’d like to start the effort of creating an Android version of the ImageJ GUI, similar to what is available on the desktop, with ImageJ2 as the processing library.

If someone is working on this already, I’d like to contribute so links to the project would be appreciated.


#5

I am also interested to hear about this. Note that you’ll find life to be much easier if you focus on the “Modern mode” UI, which is an IJ2 version of the interface. Efforts in the future will be building on the IJ2 interface, so investing efforts into porting the IJ1 interface might be largely lost in the long term. Also the IJ2 UI architecture has a more deliberate architecture with many design decisions that were made for specifically what you’re proposing: supporting alternative UI libraries/backends.

Furthermore, you may want to talk to @cmongis who has recently developed an amazing UI for ImageJ based on JavaFX.


#6

Thanks for the imagejfx reference, the GUI there is really slick. I’ll try to contact the guys regarding any gotchas they had when porting to the new UI framework.

Using the new imagej2 UI mode as the basis for the Android layout is a good point. Although I think the actual app should use UI controls typical for mobile world rather than trying to replicate the desktop look and feel(wireframes from an Android UX specialist would be beneficial too if anyone feels like volunteering) .


#7

Actually, I think Android has its own complete widget toolkit, and does not support Swing, so this would be a new UI for ImageJ2—not the Swing UI, nor the legacy one. Nor the JavaFX one probably, although I guess maybe it’s possible?

The first step will be to see whether you can even compile and/or use ImageJ2’s base components on Android at all, sans UI. I expect there will be hurdles, because we have not actually set up the build to limit things to only Android-available parts of the Java standard library.

So there is likely a lot of “under the hood” work which still needs to be done before you can go to town on the UI layer.

Help from the community on this project would be warmly welcome.


#8

android has its own ui system, I meant that the modern mode would be a basis from functional point of view (I.e. operations that are available in the existing ui and make sense on devices would be also available in the android implementation).

I’ve started working on a project in android studio IDE with the initial goal to open and display an image using imagej’s APIs. I’ll keep documenting the progress and share a github link once the first iteration is available. is this forum a good place for discussing issues with the imagej libs on android or would you suggest a different place?

One technical note - gradle is the default build system for Android studio but if it makes the potential future integration in Jenkins easier, maven can be used to build the app too. So if anyone has a strong opinion about either choice, let me know.


#9

This is the ideal place, yeah. And code-wise, we should strongly consider merging any changes you need to make to the ImageJ sources, as long as we can employ a corresponding regression testing framework to ensure things keep working on Android. The goal here is for this project—and not a fork thereof—to be Android-compatible.

I guess right now I strongly favor Maven, although if there were a suitable build system which could replace Maven, Gradle would be it. I think it is getting closer, but probably not there yet in general. That said, if we have e.g. an “imagej-android” component which does some kind of integration of lower level components and makes them friendly for Android use, I would be comfortable with specific layers like that using Gradle. But overall I think it might be confusing to heavily mix and match build systems across many components.


#10

ok, I’ll keep posting here and use Maven for the build.

Is there a set of testing images I could use to test the supported image formats?

As you mentioned above, creating an android friendly alternative for displaying images will be needed. I’ve only seen this code from imglib2-ij to do that:
ImageJFunctions.show( image );

Is there any imglib2 alternative for the display functionality, perhaps using some interfaces that could be reimplemented for Android?


#11

Can you elaborate more specifically on what you are trying to do?

Did you see the imagej-tutorials? In particular, the load-and-display-dataset tutorial?


#12

Thanks for the link, I’ve gone through the imgLib2 examples on wiki (where only ImageJFunctions.show was used) so far so the imagej display examples are helpful.

The overall goal is to see if imglib2 (and other imagej components) can be used on android as a library by other apps. As a side effect, I’d like to create a demo app that would offer similar functionality as the existing imageJ desktop application.

As the first step, I am trying to open an image stored on the device and display it in a view. I’m using the existing jars from ImageJ.app/jars as dependencies for my android project to see where it would fail.

The next step would be to try the other examples from imgLib2 wiki to locate any android incompabilities.


#13

Any Android app built with ImageJ?


#14

Nope, not to my knowledge. As I said above:

See full thread above for details.


#15

I am looking into this independently.

The first problem when trying to use ImageJ2 on android directly is the plugin loader system that relies on jar files containing plugin information in META-INF\json*.json files. However when the android app is compiled from source and referenced jar libraries, all the jars are merged and this information is lost.

One approach to solve this is to implement Android specific PluginFinder and the information from json files will need to be provided in other ways - perhaps extracting the json files from jars before the build and storing them in a resource file.


#16

Awesome!

Is it required to create uber-JARs like that on Android?

If not, I would recommend against it.

If you must do so, then you can use the SciJava utility class for combining annotations across JAR files. This will ensure the plugin annotations are combined properly.

The imagej/imagej project also has a profile deps which builds an uber-JAR of ImageJ, which properly combines the annotations in this way. You can build it with:

git clone git://github.com/imagej/imagej
cd imagej
mvn -Pdeps
ls -la target/*all.jar

#17

Short version : The merge of the jars is required on Android.

Long version: The build process on Android is a bit more complicated - .class files both from my code and all referred jars are converted into .dex files (typically only one big merged dex file or just a few of them are generated depending on the size of the app). Dex files represent custom byte code that is used by Android specific virtual machines (in the plast it was Dalvik VM and now it’s replaced by ART). All that is combined with resource files (xmls, images, configurations, …) into a single APK file that is then distributed to the end-user’s devices, typically by publishing it in Google Play store.

The whole process is described here or here for those interested in details.

Thanks for the utility for combining the annotations. I’ll try it in my project.


#18

Great. BTW, I would be happy to augment the SciJava build system to better support Android—probably via the android-maven-plugin. We could add a profile (or profiles) to the parent POM to facilitate this. If we did this, we could also have Travis CI automatically perform the Android builds to catch any Android-specific regressions early, and ensure things keep working.


#21

@ctrueden
Can you tell me more about Travis CI? i wan to have Travis CI automatically perform the Android build , what are its advantages over other android manager ?


#22

Travis CI is just a tool for performing continuous integration. Think of it as a tool for running a sequence of commands from the terminal automatically, every time you push changes to your code. It is not directly related to Android development specifically. It simply eases repetitive build tasks so you don’t have to do things manually every time.