Saving images to Byte Arrays or Streams

save
scifio
Tags: #<Tag:0x00007fd543adff28> #<Tag:0x00007fd543adfde8>

#1

I have a question similar but somehow opposite to what was considered here:

We are also trying to use SCIFIO API with Apache Spark. We would like to perform some operations in parallel on large numbers of independent images. I managed to use solutions provided in the topic above to read an image from an array of bytes, so I’m not dependent on the file system when reading - in our case files may come from HDFS (Hadoop File System), https, or even Microsoft Azure specific storage.

Previously we developed procedures to store arrays of bytes representing image files into these systems. However, the only way I see now to save an image with SCIFIO procedures is to save it to a local file. I could create a temporary file with them, read it into an array of bytes and then save it to these special systems but this is too much of a workaround. Am I missing something? Is it possible to save an image into an array of bytes (or a stream) instead of into a file?

Best regards
Mirek


Serializing multi-dimensional images to byte strings
#2

Welcome to the forum, @Miroslaw_Korzeniowsk!

SCIFIO was certainly designed with such flexibility in mind. As Mark said in the other thread you linked, there is still a pending redesign of SciJava + SCIFIO core I/O, which will make it simpler to read/write to/from non-file locations. I am very excited about this work—just strapped for time to actually finish it. But it is in the top five on my list of development priorities.

In the meantime, you should still be able to write to a byte array in memory or similar. The simplest way is probably to use LocationService#mapFile(String, IRandomAccess) to associate a particular string with a ByteArrayHandle. Then, when you pass that same string to the SCIFIO API, it should “magically” use the provided ByteArrayHandle under the hood. Definitely reply back if this does not work.

Alternately, deeper in the API, there is also Writer#setDest(RandomAccessOutputStream).

I apologize that we do not currently have a tutorial for this. If you get something working, and would care to contribute a tutorial to scifio/scifio-tutorials along those lines, that would be very welcome.


#3

Hi @ctrueden!

Thanks for your help. We have initially succeeded with writing to an array of bytes through the LocationService method.

Next week we will try to do it in a distributed way and later I will try to contribute something to the now empty Spark wiki page.

Cheers
Mirek