Skip to content

Displacements are multiscales#145

Open
jo-mueller wants to merge 5 commits into
ome:mainfrom
jo-mueller:displacements-are-multiscales
Open

Displacements are multiscales#145
jo-mueller wants to merge 5 commits into
ome:mainfrom
jo-mueller:displacements-are-multiscales

Conversation

@jo-mueller

Copy link
Copy Markdown
Contributor

This changes how displacement fields and coordinate arrays are expected to be stored in displacement/coordinate transforms, as discussed in the 06/2026 Hackathon at EMBL.

The gist: The arrays are now simply multiscale groups and the displacement/coordinate dimension should go into the channel dimension of the data.

  • updated the spec text
  • updated the examples (removed the 1d examples because 1d is not allowed anyway)
  • no change to the schema
  • updated version history

@bogovicj @clbarnes @melonora @LucaMarconato @thewtex @tischi @kevinyamauchi @jni @DragaDoncila @perlman
...who am I forgetting?

@jo-mueller jo-mueller added the enhancement New feature or request label Jun 12, 2026
@github-actions

Copy link
Copy Markdown

Automated Review URLs

@jo-mueller

Copy link
Copy Markdown
Contributor Author

Maybe should add a recommendation/statement about which resolution level of the displacement field implementations should use?

If multiple resolution levels are provided, implementations MAY choose any resolution level to compute the output.

Also, do we need to add constraints/hints on possible pathologies that can arise when downsampling vector fields/coordinate lists?

@thewtex thewtex left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇 😍 🚀

@thewtex

thewtex commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Maybe should add a recommendation/statement about which resolution level of the displacement field implementations should use?

If multiple resolution levels are provided, implementations MAY choose any resolution level to compute the output.

👍

And possibly SHOULD use the finest resolution if the result is used beyond quick visualization purposes.

Also, do we need to add constraints/hints on possible pathologies that can arise when downsampling vector fields/coordinate lists?

We could also recommend that anti-alias filtering SHOULD be applied when downsampling, similar to an intensity image multiscale.

@imagesc-bot

Copy link
Copy Markdown

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/serialise-displacement-field-as-ome-zarr/121117/6

Comment thread index.md Outdated
The coordinate dimension SHOULD be stored as the channel axis (`"name": "c"`) of the multiscale image group and have `"type": "coordinate"`.
- For `displacements` transformations, the length of the array along the `displacement` dimension (last axis) MUST equal `N`,
the number of axes in the input (and output) coordinate system. `displacements` require `M=N`.
The displacement dimension SHOULD be stored as the channel axis (`"name": "c"`) of the multiscale image group and have `"type": "displacement"`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't overload the concept of multichannel images for this purpose. A displacement isn't like a channel.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not, but we figured that until RFC3 is pushed over the line, it would be easier this way for several reasons:

  • It's not a MUST - you can also call the used axis d
  • The type "displacement" clearly indicates what it's used for
  • RFC3 will open/improve the usability of more custom axes.

It's not ideal, but solving this "properly" would turn 0.6 into containing RFC3, too.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be missing something, but I thought the concept of a "channel" axis was defined entirely w.r.t to its type field, not its name. So why is the name mentioned here? Seems like you can leave the name uncontrolled and just require the type, and then you don't need any references to "the channel axis"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, you should probably remove the arbitrary axis restrictions as soon as possible. Otherwise 5 dimensional data can't have displacement / coordinate transforms, because the resulting transform array would be 6 dimensional, and therefore invalid. You can of course work around this with dropaxis and whatnot, but given that the dimensionality restriction is completely arbitrary, I would fix that first.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like you can leave the name uncontrolled and just require the type, and then you don't need any references to "the channel axis"

That's a good suggestion I could see myself getting behind if it's a blocker for others, too. People could still name the axis c (mostly because that's what implementations expose) and as long as type: displacement is tied down that'd be good enough for me.

also, you should probably remove the arbitrary axis restrictions as soon as possible.

RFC3 is next on the menu, but that's for post-0.6.

@clbarnes clbarnes Jun 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the existing 0.6 draft, emphasis mine:

axes entries MUST be ordered by type where the time axis must come first (if present), followed by the channel * * * or custom * * * axis (if present) and the axes of type space.

From v0.5:

The order of the entries MUST correspond to the order of dimensions of the zarr arrays. In addition, the entries MUST be ordered by “type” where the “time” axis must come first (if present), followed by the “channel” * * * or custom * * * axis (if present) and the axes of type “space”.

That is, we don't currently intrinsically have a channel axis. The slot which the channel axis must occupy is also allowed to be a null or custom -typed axis. It's probably better not to normatively refer to it as a channel axis, but say that it MUST be after the time axis, MUST be before the space axes, and MUST not coincide with a channel or null or custom axis.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is, we don't currently intrinsically have a channel axis.

yeah this is my main thing. The first reference to "the channel axis" should probably not be here :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better in current state? Thanks @clbarnes for the text bricks 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants