FIJI jar dependency version collision

dependencies
jar
plugin
fiji
Tags: #<Tag:0x00007fd5432abc30> #<Tag:0x00007fd5432abac8> #<Tag:0x00007fd5432ab960> #<Tag:0x00007fd5432ab820>

#1

Hello,
I’m developing a plugin for FIJI that has a dependency already present in FIJI, but my plugin requires a more recent version (in this case mapdb-3 vs mapdb-1.0.3). The version on which my plugin depends is no loaded and thus I get a NoSuchMethodError.
What is the proper way to manage such collisions?
Thanks,
Jean


#2

Hi @jeanollion,

you can remove mapdb-1.0.3.jar from the jars folder of your Fiji installation and only keep the newer version.

AFAICT from searching for mapdb on github via search.imagej.net, mapdb is mainly required by SCIFIO, so you can test if updating mapdb to version 3.0.3 breaks anything there.

Also, for developing your plugin, I think that running maven with the -Ddelete.other.versions option should remove old versions of your dependencies:

mvn -Dimagej.app.directory=/path/to/your/Fiji.app -Ddelete.other.versions

In general, you should probably file a pull request to the SCIFIO repository, updating the mapdb.version property in pom.xml to the latest version, or even moving it over to pom-scijava where all the dependency versions are now centrally managed.


#3

Hi @imagejan,
Thank you for your answer.
If I remove the old jar, I get the plugin to work, but the updater will always try to install it. My concern is that I’d like this change to be permanent when the update site is installed, to make it easier for users.
Maybe there is a possibility to have this change permanent for my update site, using the “mark as obsolete” command (specifically for the update site) in the updater. When I tried, the updater asked to break dependency on scifio-0.27.1.jar, and then I get an error from the updater: "missing upload information for site http://update.imagej.net/ "
Is there another way, or how can I get this to work?

About the idea of making a pull request, I guess this is a decision from the developers, because the version 3 has major changes. Notably the files produced by version 1 are not compatible with version 3 it seems (headers vary between versions).


#4

Yes, sorry, my advice was wrong. Instead of simply removing the jar file, the correct way would be to start the updater and select Uninstall for the mapdb.jar. Then the updater will remember your choice for the future.

I think there is currently no way to have two incompatible versions of the same library at the same time. As a workaround, you can provide a modified library with changed package names.

But I think it would be worth discussing an update of mapdb within the SCIFIO project and also with the Bio-Formats team. --I have no idea which formats actually require that library.-- Edit: MapDB seems to be used to cache SCIFIOCell objects to disk:


#5

I did not get this to work, because of the problem I mention in my 2nd post, but instead I noticed the updater proposes an option called “shadowing”, which does exactly what I wanted: upload a version to my update site instead of another.
Thank you!
Cheers,
Jean


#6

Hello again,
I think I made a mistake in my last answer or I could not get it to work. So I tried to use the version provided of mapdb (1.0.3) but at some point it seems to be unsafe when many transactions are involved, so I’d like to use the latest version. When I try to remove the jar as you explained, Fiji doesn’t start anymore because the service that involves mapdb cannot be loaded (see console error below).
At this point what can I do ? How can I discuss the modification of MapDB version, or avoid this error at startup ?
Thanks in advance

Error while executing the main() method of class 'net.imagej.Main':
java.lang.IllegalArgumentException: Invalid service: io.scif.img.cell.cache.MapDBCache
	at org.scijava.service.ServiceHelper.createExactService(ServiceHelper.java:280)
	at org.scijava.service.ServiceHelper.loadService(ServiceHelper.java:231)
	at org.scijava.service.ServiceHelper.loadService(ServiceHelper.java:194)
	at org.scijava.service.ServiceHelper.loadServices(ServiceHelper.java:166)
	at org.scijava.Context.<init>(Context.java:279)
	at org.scijava.Context.<init>(Context.java:235)
	at org.scijava.Context.<init>(Context.java:175)
	at org.scijava.Context.<init>(Context.java:161)
	at net.imagej.ImageJ.<init>(ImageJ.java:77)
	at net.imagej.Main.main(Main.java:54)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.imagej.launcher.ClassLauncher.launch(ClassLauncher.java:279)
	at net.imagej.launcher.ClassLauncher.run(ClassLauncher.java:186)
	at net.imagej.launcher.ClassLauncher.main(ClassLauncher.java:77)
Caused by: java.lang.NullPointerException
	at java.util.ArrayList.addAll(ArrayList.java:577)
	at org.scijava.util.ClassUtils.getAnnotatedFields(ClassUtils.java:463)
	at org.scijava.util.ClassUtils.getAnnotatedFields(ClassUtils.java:433)
	at org.scijava.service.ServiceHelper.createServiceRecursively(ServiceHelper.java:316)
	at org.scijava.service.ServiceHelper.createExactService(ServiceHelper.java:269)
	... 16 more

#7

I see, this seems to be more complicated, as ImageJ itself depends on mapdb-1.0.3 via its SCIFIO dependency, and the major version 3 of mapdb might have some backwards incompatibility.

I guess the dependencies need to be updated in SCIFIO (or pom-scijava) so that you can depend on mapdb-3.0.5.jar in your own project. (@ctrueden can you comment?)


#8

Easiest way forward would be to update SCIFIO to make use of the new ImgLib2 cell image logic, which does not depend on mapdb anymore. (This work is planned regardless, but no one has time at the moment to work on it.) Then we could remove mapdb-v1 as a dependency of core ImageJ2, and downstream update sites would be free to ship what they need.


#9

Thank you for your answer. So in the long run it won’t be a problem.
As a short term solution, digging in the source code I found that setting the system variable “scijava.context.strict” to false allow FIJI to start even-though the service could not be loaded.
But I still have a problem: I can’t make work what you suggested before @imagejan

Whether I delete it from the installer or not, when I restart FIJI and the updater, it always suggest me to re-install it.
How can I fix this?
Thanks


#10

It seems I was mistaken concerning the behavior of the updater. I was under the impression that you could remove files and it would remember your choice, but apparently it doesn’t work like that (I’m not sure if it worked in the past…).

See also the discussion on this github issue where I (apparently wrongly) suggested the same:

https://github.com/openmicroscopy/bioformats/issues/2919#issuecomment-321788848


#11

To hazard a guess: I believe the Updater aggressively wants to install files that are marked as dependencies of other installed files. It might be the case that if you uninstall a dependency foo and also uninstall the only plugin p that uses foo… and nothing else depends on p nor foo… then the Updater does not bother you again about reinstalling it, at least not every time(?).


#12

Amazing that you found that! Yes, my plan has been to update the ImageJ Launcher to set this flag to false by default, so that ImageJ has fewer startup crashes. But I have not had time to do it yet.


#13

Hello Curtis,
Yes I think as mapdb-1.0.3 remains a dependency of FIJI, the updater will always try to install it if is it not present. Is there a way to “blacklist” some jar even though other jars depend on them ?


#14

Not to my knowledge.

I guess you could upload a dummy/empty version of the JAR to your own update site, so that that empty version shadows the problematic JAR file. But of course that would be a big hack. :grin:


#15

Hi Curtis,
I just tried your hack, it works perfectly. It makes the installation and update much easier for the end-user.
Thanks a lot.