Fiji and Bioformats - conflicting jars

bio-formats
fiji
Tags: #<Tag:0x00007fa30493f0e0> #<Tag:0x00007fa30493efa0>

#1

Hi all,
we encounter problems with jars that seem to conflict in our Fiji installation. Indeed we did not update our Fiji installation for a longer period of time, so we are not sure how long the problem might have been there. In detail, sometimes (!) we get errors on using methods from Loci/Bioformats to read images. In particular, the loci.formats.tiff.TiffParser claims that it cannot access a method named “readUnsignedInt()” within loci.common.RandomAccessInputStream. Checking the jar files currently being present in our Fiji installation I noticed that there are at least two jars including the class. Is it possible that both jars interfer and only one of them offers the method and the other one not? If so, this would at least explain why the error appears only sometimes since the sorting of jars in the classpath is done by random as far as I know… Or does anyone have another idea where the error might come from?

The two jars are following two:

  • jars/bio-formats/formats-common-5.1.10.jar
  • jars/bio-formats/ome-common-5.3.1.jar

The top one is delivered from the Fiji update site, the bottom one from the Java8 update site. According to the updater both are still valid and up-to-date. I wonder if something is wrong with our installation. Are there outdated jars present that should have been removed before? Or are both jars still needed? If so, does anyone have an idea how we can solve the issue, i.e. ensure that the correct method is always found?

Thanks and best regards,

Birgit


Fiji updates break plugins distributed through the Fiji update mechanism
#2

Indeed: when running Plugins > Utilities > Find Jar For Class and entering loci.common.RandomAccessInputStream, I get:

The class loci.common.RandomAccessInputStream is contained in /Users/eglinger/Fiji.app/jars/bio-formats/ome-common-5.3.1.jar
WARNING! loci.common.RandomAccessInputStream is also contained in /Users/eglinger/Fiji.app/jars/bio-formats/formats-common-5.1.10.jar!

I guess the old formats-common-5.1.10.jar should be removed from the Fiji site, but this will not be possible until the transition to Java 8 is finalized (and the whole Java-8 update site will be rendered obsolete).
(@ctrueden any comments?)

@bimoelle You can try to work around the issue by running the updater, looking for formats-common.jar, and right-click > Uninstall that file.


#3

Hi Jan,
thanks for your hint. Indeed, uninstalling the formats-common.jar locally seems to fix the issue. Anyway, a general solution would be preferable. I can see that this might be difficult due to backwards compatibility. But on the other hand, also plugins still relying on the old jar file might run into trouble if by chance the newer jar is selected from the path. Thus, seeking for a general solution would be of advantage in any case.

Best regards,

Birgit


#4

Actually, moving the old jars to the newly created Fiji-Legacy update site might solve this issue.

@ctrueden can the Bio-Formats jars be moved to Fiji-Legacy and removed from the Fiji update site?


#5

I was reluctant to do this before because:

  1. Until this thread, it seemed that having both ome-common and formats-common libraries was not causing any faulty behavior; and:
  2. Moving formats-common to Fiji-Legacy would break Bio-Formats on all existing Java 6 installations, which I did not want to do.

It was suggested on another thread to auto-enable the Fiji-Legacy site for installations running Java 6. While this is a good idea, it requires changes to the ImageJ Updater which I do not have the bandwidth to investigate. I suppose I need to make time to do it soon, however, since it seems that Bio-Formats is now already broken on Java 6 installations when both JAR files are present . :frowning:


#6

I started working on a script which will automatically enable Fiji-Legacy when the installation is using Java 6, and automatically enable Java-8 when it is using Java 8, without needing to modify the ImageJ Updater code at all. I plan to upload this script to the core Fiji update site, so that all Fiji installations start tracking the appropriate required update site.

However, the script is not yet fully working, and I need to work on other things for a while.

For reference, the current (not fully working) version of the JavaScript script is as follows:

importClass(Packages.ij.IJ)
importClass(Packages.java.lang.System)
importClass(Packages.net.imagej.updater.FilesCollection)
importClass(Packages.net.imagej.updater.util.AvailableSites)
importClass(Packages.net.imagej.updater.util.StderrProgress)
importClass(Packages.org.scijava.app.AppService)
importClass(Packages.org.scijava.log.LogService)

// NB: This script runs in ImageJ1 land, so we extract needed services.
context = IJ.runPlugIn("org.scijava.Context", "");
app = context.service(AppService.class)
log = context.service(LogService.class)

function require(name, url) {
	ijRoot = app.getApp().getBaseDirectory();
	filesCollection = new FilesCollection(log, ijRoot);
	AvailableSites.initializeAndAddSites(filesCollection, log);
	filesCollection.read();

	site = filesCollection.getUpdateSite(name, true);
	if (site == null) {
		log.error("Required update site not found: " + name);
	}
	else if (!site.isActive()) {
		log.info("Enabling required update site: " + name);
//		filesCollection.activateUpdateSite(site, new StderrProgress());
//		filesCollection.write();
	}
	else log.warn("YOU ARE ALL GOOD"); //TEMP
}

if (!"true".equals(System.getProperty("java.awt.headless"))) {
	javaVersion = System.getProperty("java.version");
	if (javaVersion.startsWith("1.6.")) require("Fiji-Legacy");
	else if (javaVersion.startsWith("1.8.")) require("Java-8");
	else {
		log.warn(
			"---------------------------------------------------------" +
			"Java version '" + javaVersion + "' is unsupported.\n" +
			"You may experience problems. See this page for guidance:\n" +
			"\n" +
			"    https://imagej.net/Java_8\n" +
			"\n" +
			"If you need assistance, please write to the ImageJ Forum:\n" +
			"\n" +
			"    http://forum.imagej.net/\n" +
			"---------------------------------------------------------");
	}
}

null

If anyone is voracious enough to make this script fully work with both Fiji-6 and Fiji-8, that would be awesome. Otherwise, I’ll come back to it within a few weeks, at which point Bio-Formats should work “out of the box” again for legacy Fiji-6 installations. In the meantime, I guess we’ll just have to keep fielding user reports about this issue (sorry, @s.besson et. al :disappointed:).


#7

I would also like to report that I get this issue when running the Grid/Collection Stitching Algorithm on an HPC cluster using the following script:

print("======================================================");
print("~~ Grid Collection / Stitching Batch ~~");

processFolder(input);

function processFolder(input) {
    list = getFileList(input);

    for (i = 0; i < list.length; i++) {
        if(File.isDirectory(input + list[i]))
            processFolder("" + input + list[i]);
            spath=input + list[i];

        if(startsWith(list[i], prefix))
            processChannel(input, list[i]);
            path = input+list[i];
    }
}

function processChannel(input, file){
    setBatchMode("hide");
    print("------------------------------------------------------");
    print("Processing: " + input + file);
    run("Grid/Collection stitching", "type=[Positions from file] order=[Defined by TileConfiguration] directory=["+input+"]layout_file=TileConfiguration.txt fusion_method=[Linear Blending] regression_threshold=0.30 max/avg_displacement_threshold=2.50 absolute_displacement_threshold=3.50 use_virtual_input_images computation_parameters=[Save memory (but be slower)] image_output=[Fuse and display]");
    setBatchMode("hide");
    imageID = getImageID();
    selectImage(imageID);
    outputdir = input + "output" + File.separator;
    File.makeDirectory(outputdir);
    saveAs("tiff", outputdir + "fused");
    print("Rendered: " + file);
}
print("======================================================");  

to throw the following errors:

There was a problem with the class loci.common.RandomAccessInputStream which can be found here:
/Fiji/1.51p/jars/bio-formats/formats-common-5.1.10.jar
/Fiji/1.51p/jars/bio-formats/ome-common-5.3.1.jar

Based on the solutions written about here, it appears that simply deleting formats-common-5.1.10.jar
as mentioned here:

[quote=“bimoelle, post:1, topic:5736, full:true”]
Hi all,
we encounter problems with jars that seem to conflict in our Fiji installation. Indeed we did not update our Fiji installation for a longer period of time, so we are not sure how long the problem might have been there. In detail, sometimes (!) we get errors on using methods from Loci/Bioformats to read images. In particular, the loci.formats.tiff.TiffParser claims that it cannot access a method named “readUnsignedInt()” within loci.common.RandomAccessInputStream. Checking the jar files currently being present in our Fiji installation I noticed that there are at least two jars including the class. Is it possible that both jars interfer and only one of them offers the method and the other one not? If so, this would at least explain why the error appears only sometimes since the sorting of jars in the classpath is done by random as far as I know… Or does anyone have another idea where the error might come from?

The two jars are following two:

  • jars/bio-formats/formats-common-5.1.10.jar
  • jars/bio-formats/ome-common-5.3.1.jar

The top one is delivered from the Fiji update site, the bottom one from the Java8 update site. According to the updater both are still valid and up-to-date. I wonder if something is wrong with our installation. Are there outdated jars present that should have been removed before? Or are both jars still needed? If so, does anyone have an idea how we can solve the issue, i.e. ensure that the correct method is always found?

Thanks and best regards,

[quote=“bimoelle, post:3, topic:5736, full:true”]
Hi Jan,
thanks for your hint. Indeed, uninstalling the formats-common.jar locally seems to fix the issue. Anyway, a general solution would be preferable. I can see that this might be difficult due to backwards compatibility. But on the other hand, also plugins still relying on the old jar file might run into trouble if by chance the newer jar is selected from the path. Thus, seeking for a general solution would be of advantage in any case.

Best regards,

will do the trick?


#8

Removing formats-common.jar has solved the problem.


#9

Together with @s.besson, we have finally—hopefully!—solved this.

The formats-common and mdbtools-java JARs now live on Fiji-Legacy only.

Now, if you have only the ImageJ and Fiji update sites enabled, you will see a popup dialog on every startup warning you to enable either the Fiji-Legacy (if using Java 6) or Java-8 (if using Java 8 or 9) update site. And if you ignore the warning and choose not to enable one of those two sites, Bio-Formats will no longer function.

If you enable both the Fiji-Legacy and Java-8 update sites, there may be continued weirdness, because you’ll get the old formats-common from Fiji-Legacy, and the new ome-common from Java-8, and they will clash as people noted above in certain circumstances. Exactly when will depend on your platform, classpath order, etc., so it’s hard to predict. But you can avoid the issue by (as before) manually uninstalling the old formats-common and mdbtools-java JARs.

I’m going to mark this issue is solved, but if anyone continues to have problems, please reply here and we’ll sort things out.


#10

Hi @ctrueden, I also have this problem. Sorry I didn’t understand the way through which you solve the problem, does the script get improved to work? Because I have around 8 conflicted versions, really have no idea what to do. Even i selected ‘ignore’ until the last one, there is no following hint to update the fiji. So confused.

Thank you for your help.
Tian


#11

@Tian_Tian It really depends on which specific JARs are conflicting. Could you please post the exact error message(s)?


#12

A post was split to a new topic: NullPointerException with Bio-Formats