Skip to content

[essimaging] Use on-the-fly wavelength lookup table for Odin #631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nvaytet merged 13 commits into from
Jun 11, 2026
Merged

[essimaging] Use on-the-fly wavelength lookup table for Odin #631

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
b7af73a
skip choppers with empty logs (probably they are not in use)
nvaytet Jun 9, 2026
7d08bb6
update notebook to use analytical chopper mode
nvaytet Jun 9, 2026
5f8b966
update tests
nvaytet Jun 9, 2026
163350d
cleanup
nvaytet Jun 9, 2026
19c1745
lint
nvaytet Jun 9, 2026
f50ed97
make analytical mode the default
nvaytet Jun 9, 2026
91b67d8
add to_disk_choppers tests
nvaytet Jun 10, 2026
23c93d3
update scippneutron version and removed unecessary deps in packages t…
nvaytet Jun 10, 2026
e4bd6d0
update lock file
nvaytet Jun 10, 2026
ef5059b
Merge branch 'main' into imaging-onthefly-lut
nvaytet Jun 10, 2026
05e6d02
do not warn when chopper dropped
nvaytet Jun 11, 2026
d1d15b2
add dask and graphviz deps back in
nvaytet Jun 11, 2026
81ed88e
update lock file
nvaytet Jun 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions packages/essdiffraction/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,11 @@ requires-python = ">=3.11"
# Make sure to list one dependency per line.
dependencies = [
"dask>=2022.1.0",

@nvaytet nvaytet Jun 10, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these dependencies are brought in by essreduce.
Removing them here avoids the issue of always keeping all the lower bounds up to date (right now, most of the lower bounds we had were lying anyway).

@jl-wynen jl-wynen Jun 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still list all direct dependencies here. Or at least direct 3rd party dependencies (dask, graphviz) to make sure we don't rely on essreduce's setup. We don't need to list lower bounds here as long as they are not tighter than essreduce's bounds.

"essreduce>=26.6.0",
"graphviz>=0.20",
"essreduce>=26.6.0",
"numpy>=2",
"plopp>=26.2.0",
"pythreejs>=2.4.1",
"sciline>=25.04.1",
"scipp>=25.11.0",
"scippneutron>=26.3.0",
"scippnexus>=23.12.0",
"tof>=25.12.0",
"ncrystal[cif]>=4.1.0",
"spglib>=2.0.0,!=2.7", # https://github.com/mctools/ncrystal/issues/320
Expand Down
72 changes: 49 additions & 23 deletions packages/essimaging/docs/odin/odin-data-reduction.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,39 @@
"metadata": {},
"outputs": [],
"source": [
"wf = odin.OdinBraggEdgeWorkflow()\n",
"wf = odin.OdinBraggEdgeWorkflow(wavelength_from=\"analytical\")\n",
"\n",
"wf[Filename[SampleRun]] = odin.data.iron_simulation_sample_small()\n",
"wf[NeXusDetectorName] = \"event_mode_detectors/timepix3\"\n",
"wf[unwrap.LookupTableFilename] = odin.data.odin_wavelength_lookup_table()"
"wf[NeXusDetectorName] = \"event_mode_detectors/timepix3\""
]
},
{
"cell_type": "markdown",
"id": "4",
"metadata": {},
"source": [
"### Patch the choppers in the file\n",
"\n",
"The chopper settings in the file contain errors. We patch them here, but note that **this step should go away in the future**."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5",
"metadata": {},
"outputs": [],
"source": [
"from ess.odin.beamline import choppers\n",
"\n",
"disk_choppers = choppers(source_position=wf.compute(Position[snx.NXsource, SampleRun]))\n",
"wf[unwrap.DiskChoppers[SampleRun]] = disk_choppers"
]
},
{
"cell_type": "markdown",
"id": "6",
"metadata": {},
"source": [
"## First look at the data\n",
"\n",
Expand All @@ -61,7 +83,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "5",
"id": "7",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -72,7 +94,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "6",
"id": "8",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -81,7 +103,7 @@
},
{
"cell_type": "markdown",
"id": "7",
"id": "9",
"metadata": {},
"source": [
"## Compute neutron wavelengths\n",
Expand All @@ -92,7 +114,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "8",
"id": "10",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -101,7 +123,7 @@
},
{
"cell_type": "markdown",
"id": "9",
"id": "11",
"metadata": {},
"source": [
"### Inspect the lookup table\n",
Expand All @@ -112,7 +134,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "10",
"id": "12",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -122,7 +144,7 @@
},
{
"cell_type": "markdown",
"id": "11",
"id": "13",
"metadata": {},
"source": [
"### Compute neutron wavelengths"
Expand All @@ -131,7 +153,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "12",
"id": "14",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -142,7 +164,7 @@
},
{
"cell_type": "markdown",
"id": "13",
"id": "15",
"metadata": {},
"source": [
"## Process the open-beam run\n",
Expand All @@ -153,19 +175,23 @@
{
"cell_type": "code",
"execution_count": null,
"id": "14",
"id": "16",
"metadata": {},
"outputs": [],
"source": [
"wf[Filename[OpenBeamRun]] = odin.data.iron_simulation_ob_small()\n",
"\n",
"# Also need to patch choppers here\n",
"wf[unwrap.DiskChoppers[OpenBeamRun]] = disk_choppers\n",
"\n",
"openbeam_wavs = wf.compute(WavelengthDetector[OpenBeamRun])\n",
"\n",
"openbeam_wavs.bins.concat().hist(wavelength=300).plot()"
]
},
{
"cell_type": "markdown",
"id": "15",
"id": "17",
"metadata": {},
"source": [
"## Select region of interest by masking outer regions\n",
Expand All @@ -177,7 +203,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "16",
"id": "18",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -186,7 +212,7 @@
},
{
"cell_type": "markdown",
"id": "17",
"id": "19",
"metadata": {},
"source": [
"The brighter areas around the edges are regions where neutrons did not travel through the sample.\n",
Expand All @@ -196,7 +222,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "18",
"id": "20",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -209,7 +235,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "19",
"id": "21",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -220,7 +246,7 @@
},
{
"cell_type": "markdown",
"id": "20",
"id": "22",
"metadata": {},
"source": [
"## Normalize to open beam\n",
Expand All @@ -232,7 +258,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "21",
"id": "23",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -248,7 +274,7 @@
},
{
"cell_type": "markdown",
"id": "22",
"id": "24",
"metadata": {},
"source": [
"## Save the final result"
Expand All @@ -257,7 +283,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "23",
"id": "25",
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -286,7 +312,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
"version": "3.12.7"
}
},
"nbformat": 4,
Expand Down
4 changes: 0 additions & 4 deletions packages/essimaging/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,6 @@ dependencies = [
"dask>=2022.1.0",
"graphviz>=0.20",
"plopp[all]>=23.10.0",
"sciline>=23.9.1",
"scipp>=25.4.0",
"scippneutron>=24.12.0",
"scippnexus>=23.11.1",
"tifffile>=2024.7.2",
"essreduce>=26.6.0",
"scitiff>=25.7",
Expand Down
1 change: 1 addition & 0 deletions packages/essimaging/src/ess/imaging/types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
RawDetector = reduce_t.RawDetector
RawMonitor = reduce_t.RawMonitor
NXdetector = snx.NXdetector
NXsource = snx.NXsource

DetectorLtotal = unwrap_t.DetectorLtotal
WavelengthDetector = unwrap_t.WavelengthDetector
Expand Down
4 changes: 2 additions & 2 deletions packages/essimaging/src/ess/odin/workflows.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def default_parameters() -> dict:


def OdinWorkflow(
wavelength_from: WavelengthLutMode = "file", **kwargs
wavelength_from: WavelengthLutMode = "analytical", **kwargs
) -> sciline.Pipeline:
"""
Workflow with default parameters for Odin.
Expand All @@ -69,7 +69,7 @@ def OdinWorkflow(


def OdinBraggEdgeWorkflow(
wavelength_from: WavelengthLutMode = "file", **kwargs
wavelength_from: WavelengthLutMode = "analytical", **kwargs
) -> sciline.Pipeline:
"""
Workflow with default parameters and masking providers
Expand Down
38 changes: 22 additions & 16 deletions packages/essimaging/tests/odin/data_reduction_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,43 +5,47 @@
import pytest
import sciline as sl
from ess import odin
from ess.odin.beamline import choppers as odin_choppers

from ess.imaging.types import (
Filename,
LookupTable,
LookupTableFilename,
NeXusDetectorName,
NXdetector,
NXsource,
OpenBeamRun,
Position,
RawDetector,
SampleRun,
WavelengthDetector,
)
from ess.reduce import unwrap


@pytest.fixture(scope="module")
def workflow() -> sl.Pipeline:
def _make_workflow(wavelength_from: unwrap.WavelengthLutMode) -> sl.Pipeline:
"""
Workflow for loading NeXus data.
"""
wf = odin.OdinBraggEdgeWorkflow()
wf = odin.OdinBraggEdgeWorkflow(wavelength_from=wavelength_from)
wf[Filename[SampleRun]] = odin.data.iron_simulation_sample_small()
wf[Filename[OpenBeamRun]] = odin.data.iron_simulation_ob_small()
wf[NeXusDetectorName] = "event_mode_detectors/timepix3"
for run_type in (SampleRun, OpenBeamRun):
wf[LookupTableFilename[run_type, NXdetector]] = (
odin.data.odin_wavelength_lookup_table()
)
# Cache the lookup table
wf[LookupTable[run_type, NXdetector]] = wf.compute(
LookupTable[run_type, NXdetector]
if wavelength_from == "file":
# Shortcut to set LookupTableFilename for both SampleRun and OpenBeamRun at once
wf[LookupTableFilename] = odin.data.odin_wavelength_lookup_table()
else:
disk_choppers = odin_choppers(
source_position=wf.compute(Position[NXsource, SampleRun])
)
# Shortcut to set DiskChoppers for both [SampleRun, NXdetector] and
# [OpenBeamRun, NXdetector] at once
wf[unwrap.DiskChoppers] = disk_choppers
return wf


@pytest.mark.parametrize("run_type", [SampleRun, OpenBeamRun])
def test_can_load_detector_data(workflow, run_type):
da = workflow.compute(RawDetector[run_type])
def test_can_load_detector_data(run_type):
wf = _make_workflow("file")
da = wf.compute(RawDetector[run_type])
assert {
"detector_number",
"position",
Expand All @@ -54,7 +58,9 @@ def test_can_load_detector_data(workflow, run_type):


@pytest.mark.parametrize("run_type", [SampleRun, OpenBeamRun])
def test_can_compute_wavelength(workflow, run_type):
da = workflow.compute(WavelengthDetector[run_type])
@pytest.mark.parametrize("wavelength_mode", ["file", "analytical"])
def test_can_compute_wavelength(run_type, wavelength_mode):
wf = _make_workflow(wavelength_mode)
da = wf.compute(WavelengthDetector[run_type])

assert "wavelength" in da.bins.coords
6 changes: 1 addition & 5 deletions packages/essnmx/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,9 @@ requires-python = ">=3.11"
# Make sure to list one dependency per line.
dependencies = [
"dask>=2022.1.0",
"essreduce>=26.6.0",
"graphviz>=0.20",
"essreduce>=26.6.0",
"plopp>=24.7.0",
"sciline>=24.06.0",
"scipp>=25.3.0",
"scippnexus>=23.12.0",
"scippneutron>=26.03.0",
"pooch>=1.5",
"pandas>=2.1.2",
"gemmi>=0.6.6",
Expand Down
2 changes: 1 addition & 1 deletion packages/essreduce/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ dependencies = [
"graphviz>=0.20",
"sciline>=25.11.0",
"scipp>=26.3.1",
"scippneutron>=26.5.0",
"scippneutron>=26.6.0",
"scippnexus>=25.06.0",
"scipy>=1.14",
]
Expand Down
Loading
Loading