Handling OME-TIFF scale metadata in napari workflows

[alandolt]

Hey,
Just wanted to bring this up for discussion. No urgency on this, I just noticed it while testing.

If you load an OME-TIFF in napari using the official napari-tiff plugin (the one that splits channels automatically), the scale metadata is applied correctly. So for example, if the scale factor is 0.5, a 1024×1024 image is shown as 512×512 in physical units, which is great because you can measure directly in µm without doing manual conversions.

The issue appears when using napari-serverkit to send the image to imaging-server-kit. The napari extension only sends the raw array and drops the metadata (accordig to my testing, maybe I've did something wrong). The server then returns the segmentation mask as a normal 1024×1024 array with no scaling info, so the mask doesn’t match the scaled display of the original image in napari.

So the question is: how should we handle the scaling?

Option A: Have the napari extension also send the metadata (including scale). The imaging server would then keep track of it and send back the mask with the correct scale info, which napari could apply (e.g. using the scale parameter in add_labels()).

Option B: Handle it entirely on the napari client side. Meaning: keep the mask and the displayed layer aligned locally, without involving the server.

I’m curious what feels like the cleaner approach to others here.

[MalloryWittwer]

Hi @alandolt, thanks for noticing that. Indeed, for now we only send the pixels, without any of the display metadata (or file metadata) from the input image for processing. In your segmentation example, I don't see an obvious way to reconstruct the display metadata from Napari on the server side.

I think the easiest might be, as you suggest, to go with Option B, perhaps by implementing a little event handler that would react to a new layer being added, or its data updated, and sync the "scale" parameter accordingly.

Note that with the server kit, you can return info about display metadata, including how the scaling should be set in Napari. For example, the code below takes the Z scale as input and applies it to the returned mask (and image).

import imaging_server_kit as sk
import skimage.data
import napari

@sk.algorithm(
    parameters={
        "image": sk.Image(),
        "z_scale": sk.Float(name="Z scale", min=1, default=5, step=1, auto_call=True),
    }
)
def scaling_test(image, z_scale):
    scale_meta = {"scale": (z_scale, 1, 1)}
    return (
        sk.Image(image, meta=scale_meta),
        sk.Mask(image > image.mean(), meta=scale_meta),
    )

if __name__ == "__main__":
    viewer = sk.to_napari(scaling_test)
    viewer.add_image(skimage.data.brain())
    napari.run()

scale_test.mp4

However, I understand that this doesn't exactly address the issue you're describing. I will let you know in case I can think of another possibility!