AVIF decoding on Linux ARM64 introduces green blocks, affects >=0.34.0

[fraunos]

Possible bug

Is this a possible bug in a feature of sharp, unrelated to installation?

• Running npm install sharp completes without error.
• Running node -e "require('sharp')" completes without error.

If you cannot confirm both of these, please open an installation issue instead.

Are you using the latest version of sharp?

• I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.

If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

This does not happen on 0.33.5, but seemingly all versions above.

System:
OS: Linux 6.18 Manjaro Linux
CPU: (14) x64 Intel(R) Core(TM) Ultra 5 235U
Memory: 4.61 GB / 15.07 GB
Container: Yes
Shell: 4.3.3 - /usr/bin/fish
Binaries:
Node: 22.15.0 - /run/user/1000/fnm_multishells/2575629_1769414932838/bin/node
Yarn: 1.22.22 - /usr/bin/yarn
npm: 10.9.2 - /run/user/1000/fnm_multishells/2575629_1769414932838/bin/npm
pnpm: 10.28.0 - /usr/bin/pnpm
npmPackages:
sharp: ~0.35.0-rc.0 => 0.35.0-rc.0

System:
OS: Linux 5.15 Ubuntu 20.04.5 LTS (Focal Fossa)
CPU: (4) arm64 Neoverse-N1
Memory: 13.48 GB / 15.41 GB
Container: Yes
Shell: 5.0.17 - /bin/bash
Binaries:
Node: 22.22.0 - /opt/sdk/node_v22_aarch64/bin/node
npm: 10.9.4 - /opt/sdk/node_v22_aarch64/bin/npm
npmPackages:
sharp: ~0.35.0-rc.0 => 0.35.0-rc.0

Does this problem relate to file caching?

The default behaviour of libvips is to cache input files, which can lead to EBUSY or EPERM errors on Windows.
Use sharp.cache(false) to switch this feature off.

• Adding sharp.cache(false) does not fix this problem.

Does this problem relate to images appearing to have been rotated by 90 degrees?

Images that contain EXIF Orientation metadata are not auto-oriented. By default, EXIF metadata is removed.

• To auto-orient pixel values use the parameter-less rotate() operation.

• To retain EXIF Orientation use keepExif().

• Using rotate() or keepExif() does not fix this problem.

What are the steps to reproduce?

1. Take sharp version greater or equal 0.34.0
2. Take any image
3. Encode AVIF or HEIF AV1 on x64 architecture machine
4. Take x64 encoded image and move to arm64 architecture machine
5. Encode any image on arm64 arch
6. Encoded image is green

What is the expected behaviour?

Image decodes without color glitches.

Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this problem

(async () => {
    const sharp = require('sharp');

    const testImage = '2569067123_aca715a2ee_o.jpg'

    console.log(sharp.versions)

    if (process.arch === 'x64') {
        await sharp(testImage).heif({ compression: 'av1' }).toFile('test_image_x86.avif')
        // or .avif(), seems equally broken
        // await sharp(testImage).avif().toFile('test_image_x86.avif')
        await sharp('test_image_x86.avif').jpeg().toFile('output/test_image_x86_x86.jpeg')
    }

    if (process.arch === 'arm64') {
        await sharp(testImage).heif({ compression: 'av1' }).toFile('test_image_arm.avif')
        await sharp('test_image_arm.avif').jpeg().toFile('output/test_image_arm_arm.jpeg')
        await sharp('test_image_x86.avif').jpeg().toFile('output/test_image_x86_arm.jpeg')
    }

})()

Please provide sample image(s) that help explain this problem

x64 to x64

arm64 to arm64

x64 to arm

[fraunos]

Fun fact - you can see when encoding was done on single thread

[lovell]

Thanks for reporting, there have been a few previous examples of dark edges turning into green blocks when decoding AVIF images, but this is probably the most extreme I've seen.

Are you able to provide the intermediate test_image_x86.avif and test_image_arm.avif images (perhaps as a zip file so GitHub doesn't try to process them)?

Please can you try altering the effort and quality avif properties to see if that makes any difference.

I'll try to reproduce locally.

[lovell]

I've been able to reproduce locally via QEMU. This looks like a problem with the AV1 chroma-from-luma prediction at decode time, possibly due to a (NEON?) bug in the underlying aom library.

The problem get progressively worse with input AVIF images encoded at a higher effort i.e. encoded at decreasing speed. The problem starts to get less bad at effort=5 (speed=4), maybe due to a different prediction method. The worst is effort=4 (speed=5), which happens to be the default in sharp.

The problem goes away entirely when using chroma-subampling (chromaSubsampling: 4:2:0) as this prevents chroma-from-luma.

The next step is to reproduce this using the aomdec command line tool, without sharp/libvips/libheif.

[lovell]

I've had no joy reproducing with anything other than the prebuilt binaries provided by sharp so my best guess is that we've hit an ARM64 gcc compiler bug e.g. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118976

The current binaries are built on the EOL gcc 10 but we're about to upgrade to gcc 15 so let's wait for that.

[kleisauke]

@lovell It may also be worth testing the musl-based Linux arm64 binaries, which are built with GCC 10.3.1 rather than GCC 10.5.0.

[lovell]

Alpine Linux on ARM64 exhibits the same problem with the current prebuilt binaries.

[kleisauke]

In that case, we should probably also update the Alpine base image. According to https://endoflife.date/alpine-linux, version 3.20 (GCC 13.2.1, musl 1.2.5) would be a sensible choice.

As an aside, we usually keep these toolchain versions aligned with https://github.com/nodejs/node/blob/main/BUILDING.md, but the Alpine version listed there is outdated. I've opened a PR¹ to update it.

Footnotes

1. https://github.com/nodejs/node/pull/61569 ↩

[fraunos]

Thanks for quick troubleshooting! Waiting for the next version 🤩

[fraunos]

Hi! Will the Release Candidate version with this bugfix be available? How is the work on 0.35.0 going overall?