Issues with the 2017-05-30 update on the Java-8 site

#1

Hi everyone especially @ctrueden,

i was investigating issues with the automated deployment of the Bio-Formats update site (see recent Travis builds). The errors started on the 31th of May and are due to a conflict between formats-gpl and the batik JARs.

While trying to debug further, I realized there was a large JAR update on the 30th of May to the Java8 update site. This update modified batik but also included the deployment of an ome-xml-5.5.2.jar under jars/bio-formats (see http://sites.imagej.net/Java-8/jars/bio-formats/). As this version of ome-xml does not exist yet, I suspect it was the result of either a manual workflow or an automated script which might need updates.

Does anyone know more about this deployment and how to resolve these issues (batik conflict and invalid JAR)?

Best,
Sebastien

#2

The org.openmicroscopy:ome-xml:5.5.2 release has apparently existed since Tue 28 Feb 2017:

Just for ease of reading for anyone else following this, the error in question is:

File 'jars/bio-formats/formats-bsd-5.5.2-20170529.105731-2.jar':
Depends on jars/batik.jar which is about to be removed.
Resolutions:
  1 Break the dependency
  2 Replace with dependency to jars/bio-formats/formats-gpl-5.5.2-20170607.201839-4.jar
  3 Replace all dependencies on jars/batik.jar with jars/bio-formats/formats-gpl-5.5.2-20170607.201839-4.jar

The problem was, I believe, due to the fact that the Bio-Formats update site’s metadata had bio-formats-bsd flagged as depending on batik. The batik JAR is no longer shipped by the core Fiji update site; it moved to Fiji-Legacy. This was done because it is an uber-JAR which causes endless trouble with dependency overlap and detection. Fortunately, the Batik project has since split its library into (mostly) well-organized components, which we now ship on the Java-8 update site.

To address the issue, I manually purged the batik (uber-JAR) dependency from formats-bsd on the Bio-Formats update site. I hope that the next time Travis runs, this error no longer occurs. It is possible there will be further errors, but we can investigate those as they arise.

#3

Hi Curtis,

Many thanks for fixing the formats-bsd metadata. Out of curiosity, what is the best documentation page explaining how to inspect and fix this metadata?. I think this might have been some legacy behavior but as there should be no more batik dependency in Bio-Formats, I hope this will resolve the issue permanently.

On the ome-xml front, you are correct, I got confused by the version numbers but this component has been released to Central. As the latest ome-xml releases mostly contain documentation and C++ patches, we haven’t bumped the dependency in Bio-Formats although it should be completely harmless. From https://github.com/scijava/pom-scijava/commit/c97667404b8b0da5c2e2f7429f80e1f7f822cd89 I assume the pom-scijava strategy is to bump all these libraries to the latest release on Central. In this case, it might be probably a good idea for us to bump internally so that we don’t create confusion.

Sebastien

#4

Following up on my previous answer, I can confirm that a re-run of the Travis build now uploaded the latest JARs to the update site. Marking this thread as fixed.

#5

There is not really much documentation. The only thing that even comes close, to my knowledge, is the “Advanced mode” section of the Updater page which reads:

The details for each component can be edited by writing below the respective entry.

This means that you can literally edit that list of dependencies using that big text field in the graphical updater’s Advanced mode.

However, this is not how I fixed the formats-bsd.jar's dependency on the old batik.jar. To do that, I edited the Bio-Formats update site’s db.xml.gz file directly to purge the dependencies. This is honestly easier, but of course: 1) requires admin access to the sites.imagej.net box, and 2) must be done extremely carefully to avoid ending up with invalid XML.

It would be nice to have A) better documentation for the Updater; and B) more powerful Updater-related CLI tools. But no one has time/energy/expertise/funding/etc. to work on this at the moment.

In general, yes. But it’s most important that the BOM list compatible versions. So if, e.g., a new version of ome-xml is released which breaks backward compatibility, and some Bio-Formats artifacts have not yet been updated to work correctly with it, we would of course hold off on updating the ome-xml version in pom-scijava until such time as all components in the dependency tree are realigned.