Struggling with using blosc compression

[jonblower]

This probably isn't a JZarr problem per se, but I'm struggling to get my Java program to recognise my native c-blosc library.

I'm on an M1 Mac (which might be significant), using JDK17 with NetBeans, using maven as a build system. The c-blosc library was installed with Homebrew.

I've written a simple program and am trying to run it using the command-line switch -Djna.library.path=/opt/homebrew/Cellar/c-blosc/1.21.1/. I only need blosc at runtime, not compile time.

Whatever I try, I end up with:
Exception in thread "main" java.lang.UnsatisfiedLinkError: /private/var/folders/s2/_3zsgdqd6pj0y20x4chv8g640000gn/T/jna-105417/jna11531817638949407677.tmp: dlopen(/private/var/folders/s2/_3zsgdqd6pj0y20x4chv8g640000gn/T/jna-105417/jna11531817638949407677.tmp, 1): no suitable image found.

Are there any more details you can provide about how to set this up in a maven-based system?

I'm wondering whether there may be an issue with the M1 processor (which is ARM based). The output of java -version is:

openjdk version "17.0.1" 2021-10-19 LTS
OpenJDK Runtime Environment Zulu17.30+15-CA (build 17.0.1+12-LTS)
OpenJDK 64-Bit Server VM Zulu17.30+15-CA (build 17.0.1+12-LTS, mixed mode, sharing)

[sbesson]

@jonblower we encountered the same issue as yours in the reader of the Bio-Formats library which consumes jzarr - see ome/ZarrReader#16.

A quick investigation seems to reveal this is related to the old version of the net.java.dev.jna:jna which is transitively shipped by org.lasersonlab:jblosc. In my local testing overriding this version to use 5.10 was sufficient to fix the issue on arm64. It would be interesting to see if that's sufficient in your use case as well.

[jonblower]

@sbesson Thanks very much! Including version 5.10 of the JAR fixed the problem. (Also I realised that I needed to add lib/ to the end of the command line switch, i.e. -Djna.library.path=/opt/homebrew/Cellar/c-blosc/1.21.1/lib/.)

[SabineEmbacher]

Great!

[SabineEmbacher]

Possibly interesting for you...

I plan to extend the library in such a way that it is possible for users to add any compressors at runtime, using Java SPI (Service Provider Interface), and thus also to be able to decide for themselves whether these are included via JNI or in another way.

[sbesson]

@SabineEmbacher would you considering to release a new version of jzarr with a more recent version of net.java.dev.jna:jna to facilitate the usage on OSX/arm64?

This incidentally raises the question of the Java interface to the Blosc library since org.lasersonlab:jblosc hasn't been released in almost 4 years. Do you know whether there would be alternatives?

[SabineEmbacher]

@sbesson

@SabineEmbacher would you considering to release a new version of jzarr with a more recent version of net.java.dev.jna:jna to facilitate the usage on OSX/arm64?

Of corse, yes!
However, the implications are not clear to me yet.
I am not familiar with the newer version of net.java.dev.jna:jna.

This incidentally raises the question of the Java interface to the Blosc library since org.lasersonlab:jblosc hasn't been released in almost 4 years. Do you know whether there would be alternatives?

When I have to use a C library in Java, I always have some kind of stomach ache.
I have experienced several times that an error in a connected C library can crash the entire Java VM without a meaningful error message.
So my first thought is: "if no alternative can be found, couldn't the Blosc algorithm perhaps be implemented directly in Java? Would there be someone who would want to do this together with me?"

Or is it really that hard to implement it in pure Java that we have to call it "Mission Impossible"?

I wish you all a great day!

[jonblower]

A pure-Java implementation of blosc would be a great contribution to the community! But it looks like quite large effort. If I'm understanding the docs correctly, blosc supports a large number of underlying codecs. So I guess one would need to not only implement the blocking algorithm, but also the codecs. This kind of low-level library is not my forte I'm afraid!