From ff3fab18c73f80135cce87c74ef0161ed49517c3 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:03:53 +0200 Subject: [PATCH 01/16] Add X-ray CW polarization optics ADR --- .../xray-cw-polarization-optics.md | 449 ++++++++++++++++++ 1 file changed, 449 insertions(+) create mode 100644 docs/dev/adrs/suggestions/xray-cw-polarization-optics.md diff --git a/docs/dev/adrs/suggestions/xray-cw-polarization-optics.md b/docs/dev/adrs/suggestions/xray-cw-polarization-optics.md new file mode 100644 index 000000000..6a84aefa2 --- /dev/null +++ b/docs/dev/adrs/suggestions/xray-cw-polarization-optics.md @@ -0,0 +1,449 @@ +# ADR: X-ray CW Polarization Optics + +## Status + +Proposed. + +## Date + +2026-06-17 + +## Group + +Experiment model. + +## Context + +This ADR follows the conventions in +[`AGENTS.md`](../../../../AGENTS.md). + +Constant-wavelength X-ray powder calculations need the same +Lorentz-polarization controls that FullProf exposes on the PCR +`Lambda1 Lambda2 Ratio Bkpos Wdt Cthm muR AsyLim Rpolarz 2nd-muR` +line. The two relevant values are: + +| FullProf field | Backend field | Meaning | +| -------------- | ------------- | ------- | +| `Rpolarz` | Cryspy `_setup_K` | Polarization coefficient in the CW Lorentz-polarization factor. | +| `Cthm` | Cryspy `_setup_cthm` | `cos²(2θm)`, where `2θm` is the pre-specimen monochromator angle. | + +These names are calculator-oriented and should not become the public +EasyDiffraction API. The IUCr powder dictionary uses the physical +monochromator angle: + +- `_pd_instr.2theta_monochr_pre` +- `_pd_instr.monochr_pre_spec` + +The core dictionary has polarization provenance fields: + +- `_diffrn_radiation.polarisn_ratio` +- `_diffrn_radiation.polarisn_norm` + +However, `_diffrn_radiation.polarisn_ratio` is not an exact alias for +FullProf `Rpolarz` / Cryspy `K`, so the first implementation persists +the EasyDiffraction polarization coefficient under the experiment-tier +short instrument namespace (`_instr.*`, like every other instrument +field) rather than silently claiming strict IUCr equivalence. The IUCr +items above are recorded as provenance and become report-export +candidates, consistent with +[`iucr-cif-tag-alignment.md`](../accepted/iucr-cif-tag-alignment.md) +(experiment tier keeps short UX names in the default save; IUCr/ +`_easydiffraction_*` forms are a report-export concern). + +### Backend support already present + +The Cryspy `Setup` item already declares both fields as optional, +non-refinable descriptors (`C_item_loop_classes/cl_1_setup.py`): +`k` (CIF `_setup_K`) and `cthm` (CIF `_setup_cthm`), with defaults +`k = 0.0` and `cthm = 0.91`. Neither is in Cryspy's refinable-attribute +set, so binding them is a value-injection task, not a backend +extension. (Note the FullProf diagnostic below uses `Cthm = 0.8`, which +differs from Cryspy's `0.91` default — defaults must be set explicitly, +not inherited from the backend.) CrysFML's current Python wrapper has no +equivalent declared field (see Decision 5). + +The existing CW instrument category is a single class, +`CwlPdInstrument` (factory tag `cwl-pd`, +`src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py`), +serving **both** neutron and X-ray, **both** `bragg` and `total` +scattering, across the `cryspy`, `crysfml`, and `pdffit` calculators. +The recent second-wavelength fields (`setup_wavelength_2`, +`setup_wavelength_2_to_1_ratio`) on `CwlInstrumentBase` are the closest +precedent for adding fixed, non-refinable instrument descriptors and are +the model this ADR follows for declaration style. + +### Diagnostic evidence + +`docs/docs/verification/pd-xray-pbso4-single-polarized-wdt48.py` +adds a focused diagnostic reference generated from +`pbsox_single_polarized_wdt48.pcr`: + +- `Lambda1 = Lambda2 = 1.540560` +- `Ratio = 0` +- `Wdt = 48` +- `Rpolarz = 0.5` +- `Cthm = 0.8` + +The current EasyDiffraction adapters do not expose or bind these +optics fields. Against that FullProf reference the current results are: + +| Engine | State | Profile diff | Max deviation | Area ratio | Correlation | +| ------ | ----- | ------------ | ------------- | ---------- | ----------- | +| cryspy | raw | 33.65 % | 33.54 % | 0.7537 | 0.9931 | +| cryspy | scale + U/V/W/Y refined | 10.20 % | 6.48 % | 1.1222 | 0.9950 | +| crysfml | raw | 36.18 % | 34.63 % | 0.6859 | 0.9895 | +| crysfml | scale + U/V/W/Y refined | 10.08 % | 6.69 % | 1.1156 | 0.9951 | + +The large raw area mismatch is expected: the FullProf reference contains +an active X-ray polarization correction, while EasyDiffraction currently +uses the backend defaults. The residual ≈10 % profile difference after +profile refinement is not assigned solely to polarization because the +PbSO4 X-ray diagnostics also show anomalous-dispersion table differences +between FullProf and Cryspy/CrysFML. + +## Decision + +### 1. Keep the fields in `experiment.instrument` + +These values describe incident-beam optics and the diffractometer +monochromator, not a peak-shape model. They belong on the CW powder +instrument category. This matches the rejection of a separate optics +category (see Alternatives). + +### 2. Split CW powder instruments by radiation probe + +The fields are useful for X-ray CW powder experiments and meaningless +for neutron CW powder experiments, where the polarization coefficient is +identically zero (unlike sample absorption, which is physically real for +both probes — so the +[`model-sample-absorption.md`](../accepted/model-sample-absorption.md) +single-shared-category precedent does not transfer cleanly here). Use the +existing `Compatibility.radiation_probe` axis and register separate +instrument classes: + +- `CwlPdNeutronInstrument` +- `CwlPdXrayInstrument` + +`CwlPdXrayInstrument` owns the X-ray optics fields. The neutron class +does not expose them. + +**This is the first use of `radiation_probe` as a category-routing +discriminator in the codebase.** It therefore carries concrete, +non-trivial impact that the implementing plan must cover: + +- `InstrumentFactory` default rules + (`.../instrument/factory.py`) currently key only on + `(beam_mode, sample_form)`. They must gain `radiation_probe`, and the + single `cwl-pd` tag splits into `cwl-pd-neutron` / `cwl-pd-xray` + (factory tags follow + [`factory-tag-naming.md`](../accepted/factory-tag-naming.md)). +- The call site `BraggPdExperiment` (`.../experiment/item/bragg_pd.py`) + builds the default instrument tag via + `InstrumentFactory.default_tag(scattering_type, beam_mode, + sample_form)` and must also pass `radiation_probe`. +- Renaming the persisted **instrument** tag is a breaking change to + saved projects and to any test/tutorial CIF pinning the instrument + `cwl-pd`. The project is in beta (no legacy shims), so the rename is + acceptable. The plan should sweep with `git grep -n 'cwl-pd'` but + **classify** the hits: only the instrument-category tag + (`instrument/cwl.py`, `instrument/factory.py`, and instrument-tag + references in `analysis/calculators/support.py` plus the instrument + factory/support tests) is retired. The string `cwl-pd` is **also** an + unrelated tag on the data-range category (`data_range/cwl.py:48`, + `data_range/factory.py:21`, and `data_range/test_factory.py`); that is + axis/range metadata, not incident-beam optics, and **must be left + unchanged** — this ADR does not rename CW-powder category tags + generally. + +**Scope is powder-Bragg CW only, and total scattering needs no +migration.** Only two call sites instantiate an instrument through +`InstrumentFactory.default_tag(...)`: `BraggPdExperiment.__init__` +(`.../experiment/item/bragg_pd.py:61`) and `ScExperimentBase.__init__` +(`.../experiment/item/base.py:505`). `TotalPdExperiment` +(`.../experiment/item/total_pd.py`) delegates to `PdExperimentBase` and +**never builds an instrument via the factory**, so the `total`/`pdffit` +PDF path does not route through `cwl-pd` and is untouched by the split. +(The `TOTAL`/`PDFFIT` entries in `CwlPdInstrument.compatibility` are +unused aspirational metadata, not a live routing path.) The routing +matrix the plan must implement: + +| scattering | beam_mode | sample_form | radiation_probe | default tag | +| ---------- | --------- | ----------- | --------------- | ----------- | +| bragg | constant wavelength | powder | neutron | `cwl-pd-neutron` | +| bragg | constant wavelength | powder | xray | `cwl-pd-xray` | +| bragg | constant wavelength | single crystal | (any) | `cwl-sc` (unchanged) | +| bragg | time of flight | powder | (any) | `tof-pd` (unchanged) | +| total | constant wavelength | powder | (any) | no factory instrument (unchanged) | + +Single crystal (`cwl-sc`) and TOF (`tof-pd`/`tof-sc`) stay +probe-neutral: this ADR scopes X-ray optics to powder-Bragg CW, where +the FullProf LP line lives. The `InstrumentFactory` rule for the +powder-Bragg CW slot is the only one that gains a `radiation_probe` +discriminator, and the **instrument** `cwl-pd` tag is the only retired +tag — the identically named data-range tag is out of scope (see the +classification note above). + +**Neutron-neutral defaults — no change to existing results until +opt-in.** Even on `CwlPdXrayInstrument`, the polarization coefficient +defaults to `0.0` (and the monochromator term is then inert, since the +Lorentz-polarization factor reduces to the neutron form when `k = 0` — +see Decision 4). Existing X-ray verification numbers therefore do **not** +change until a user explicitly sets the coefficient. Picking a non-zero +characteristic-radiation default is deferred (see Deferred Work). + +### 3. Use physical public names, non-refinable + +Expose two fixed, non-refinable `NumericDescriptor`s (mirroring the +second-wavelength fields — no `Parameter.free` flag, so the fitter never +touches them): + +```python +experiment.instrument.setup_polarization_coefficient +experiment.instrument.setup_monochromator_twotheta +``` + +Do not expose public names `setup_k`, `setup_cthm`, `K`, or `Cthm`. + +`setup_polarization_coefficient` is a dimensionless fixed descriptor. +It maps directly to FullProf `Rpolarz` and Cryspy `_setup_K`. It is a +polarization fraction, so the value is constrained to `[0, 1]` and +**defaults to `0.0`** (no correction — the pure opt-in default of +Decision 2): + +```python +NumericDescriptor( + name='polarization_coefficient', + units='', + value_spec=AttributeSpec( + default=0.0, + validator=RangeValidator(ge=0.0, le=1.0), + ), + tags=TagSpec( + edi_names=['_instrument.setup_polarization_coefficient'], + cif_names=['_instr.polarization_coefficient'], + ), +) +``` + +`setup_monochromator_twotheta` is a fixed descriptor in degrees. Its +IUCr provenance is the physical item `_pd_instr.2theta_monochr_pre`. It +**defaults to `0.0` degrees**, which the conversion in Decision 3 +(below) maps to `cthm = cos²(0) = 1.0` — i.e. "no pre-sample +monochromator", the correct neutral starting point (and a no-op while +the coefficient is `0.0`). The value is constrained to `[0, 180)` +degrees: + +```python +NumericDescriptor( + name='monochromator_twotheta', + units='degrees', + value_spec=AttributeSpec( + default=0.0, + validator=RangeValidator(ge=0.0, lt=180.0), + ), + tags=TagSpec( + edi_names=['_instrument.setup_monochromator_twotheta'], + cif_names=['_instr.monochromator_twotheta'], + ), +) +``` + +Neither field inherits a value from the Cryspy backend defaults +(`k = 0.0`, `cthm = 0.91`); both are set explicitly here so an opt-in +user gets the EasyDiffraction default, not a hidden backend value. The +`@typechecked`/`RangeValidator` boundary rejects out-of-range or +wrong-type user input (a `typeguard.TypeCheckError` for type, a range +error for value), keeping the user-input edge case explicit. + +Backend adapters convert the public angle to the backend value: + +```python +cthm = cos(radians(setup_monochromator_twotheta)) ** 2 +``` + +because the public value is the monochromator `2θ` angle and the +backend wants `cos²(2θm)`. This mirrors the established public→backend +conversion pattern (`_march_r_to_cryspy_g1` in +`analysis/calculators/cryspy.py`, where the public March coefficient is +inverted before it reaches Cryspy). + +### 4. Bind Cryspy directly + +The Cryspy CW Lorentz factor is: + +```python +hh = 1 - k + k * cthm * cos(two_theta) ** 2 +lorentz = hh / (sin(theta) * sin(two_theta)) +``` + +where `k = 0` for neutrons, `k = 0.5` for characteristic X-ray +radiation, and `cthm = cos²(2θm)`. Note `k = 0` collapses `hh` to `1`, +which is why the neutron path and the as-yet-unset X-ray path are both +no-ops. + +Because Cryspy already declares `k`/`cthm` (see Context), the Cryspy +adapter only needs to inject values for `CwlPdXrayInstrument`: + +- emit `_setup_K` and `_setup_cthm` in the generated CIF; +- patch the cached Cryspy dictionaries with `k` and `cthm` (the same + cached-dict update path used for other instrument scalars) so + minimizer iterations do not recompute against stale values; +- include these fields in the cache-invalidation surface. + +Cryspy thus applies the LP factor **internally**, on its own +`two_theta` grid — it does **not** consume the EasyDiffraction `hh` +multiplier at runtime. The only EasyDiffraction code on the Cryspy path +is the angle→`cthm` conversion (the input it is fed). See Decision 5 for +exactly what is shared and what is not. + +### 5. Bind CrysFML through the CFL path or a shared adapter correction + +The current CrysFML Python wrapper exposes `patterns_simulation(strings)` +around a CFL parser, and the adapter maps instrument scalars through +`_INSTRUMENT_ATTRIBUTE_MAP` in `analysis/calculators/crysfml.py`. The +bundled CFL examples document `LAMBDA`, `UVWXY`, `ASYM`, `WDT`, and +`Zero_Sy`, but they do not show `Cthm` or `Rpolarz` condition lines. + +Implementation must first verify whether the active CrysFML CFL grammar +accepts a native polarization/monochromator line. If it does, the +CrysFML adapter should emit that line. + +If no native CFL input exists, the CrysFML adapter applies the same +pointwise multiplier as Cryspy's `hh` term after CrysFML returns the +powder pattern: + +```python +hh = 1 - k + k * cthm * cos(two_theta) ** 2 +y_corrected = hh * y_crysfml +``` + +This fallback is acceptable because it is a smooth +Lorentz-polarization envelope on the calculated CW powder pattern. + +**What is shared vs. what is not.** Cryspy binds natively (Decision 4) +and CrysFML uses the fallback multiplier, so the two engines do **not** +run the same multiplier code path. The shared surface is therefore split +into two precisely scoped, unit-tested pieces, not one "helper consumed +by both adapters": + +1. **Angle→`cthm` conversion** — a single + `monochromator_cthm(twotheta_deg) -> cos²(radians(twotheta))` helper + used by **both** adapters to derive the backend input (Cryspy emits + the result as `_setup_cthm`; the CrysFML fallback feeds it into the + envelope). This is the only runtime code shared by both paths. +2. **LP envelope oracle** — a single + `lp_factor(two_theta, k, cthm) -> hh` reference implementing + `hh = 1 - k + k·cthm·cos²(2θ)`. It is the CrysFML fallback's runtime + multiplier **and** the oracle a verification test uses to assert that + Cryspy's native `_setup_K`/`_setup_cthm` output matches the same + formula. Cryspy's runtime path stays native; the oracle guards + against the two engines drifting without injecting EasyDiffraction + code into Cryspy. + +This keeps a single source of truth for the formula (the absorption +shared-envelope discipline of +[`model-sample-absorption.md`](../accepted/model-sample-absorption.md), +Decision 5) while honoring Cryspy's native binding. + +### 6. Keep Wdt separate + +`Wdt` is a peak-window/truncation setting, not beam optics. It should be +handled by a separate open issue for CrysFML peak-shape window control, +not by this ADR. No such issue file exists yet under +`docs/dev/issues/open/`; the implementing plan should file one so the +deferral is tracked rather than lost. + +## Concrete files likely to change + +- `src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py` + — split `CwlPdInstrument` into neutron/X-ray classes; declare the two + descriptors and their properties on the X-ray class. +- `.../instrument/factory.py` — add `radiation_probe` to the default + routing rules; new tags `cwl-pd-neutron` / `cwl-pd-xray`. +- `.../instrument/__init__.py` — register the new concrete classes + (every concrete class is imported to trigger `@Factory.register`). +- `src/easydiffraction/datablocks/experiment/item/bragg_pd.py` — pass + `radiation_probe` into `InstrumentFactory.default_tag(...)`. +- `src/easydiffraction/analysis/calculators/cryspy.py` — emit `_setup_K` + / `_setup_cthm` (via the shared angle→`cthm` helper), patch cached + dicts; native LP, no runtime envelope. +- `src/easydiffraction/analysis/calculators/crysfml.py` — CFL line or the + shared `lp_factor` post-pattern multiplier. +- A new shared module (alongside the existing calculator helpers) + holding the two pieces from Decision 5: `monochromator_cthm` (angle→ + `cthm`, used by both adapters) and `lp_factor` (the `hh` envelope; the + CrysFML fallback multiplier and the Cryspy-vs-formula test oracle). +- Tests/tutorials/docs referencing the `cwl-pd` tag. + +## Consequences + +- Neutron CW powder users do not see inactive X-ray optics fields. +- X-ray CW powder users can reproduce FullProf/Cryspy + Lorentz-polarization inputs with discoverable names. +- The codebase gains its first `radiation_probe`-routed category, and + the persisted CW-powder instrument tag changes from `cwl-pd` to + probe-specific tags (a beta-acceptable breaking change). +- Existing X-ray results are unchanged until a user opts in by setting a + non-zero polarization coefficient. +- Project persistence stores the physical monochromator angle and the + polarization coefficient under short `_instr.*` tags; the IUCr items + (`_pd_instr.2theta_monochr_pre`, `_diffrn_radiation.polarisn_ratio`) + are recorded as provenance and remain report-export candidates. +- Verification remains split: optics binding must be tested separately + from the known X-ray scattering-table discrepancy. + +## Open Questions + +- **Split vs. shared category.** Splitting `CwlPdInstrument` by probe is + the first `radiation_probe` routing and the larger refactor; the + alternative (one shared class, fields inert for neutrons via the + `k = 0` default) is a smaller change but clutters the neutron surface + with always-zero X-ray knobs. This ADR recommends the split because + the fields are physically inapplicable (not merely defaulted-off) for + neutrons; reviewers should confirm this is worth the routing + precedent. See Alternatives. +- **Default X-ray coefficient.** Keep the default at `0.0` (pure opt-in, + no result change) until an explicit incident-source model exists, or + pick `0.5` for characteristic radiation now? Recommendation: keep + `0.0` until the source model lands (see Deferred Work). + +## Alternatives Considered + +### Expose `setup_k` and `setup_cthm` + +Rejected. These names are backend/PCR implementation details and are +not discoverable for non-programmer users. + +### Keep one `CwlPdInstrument` with neutron-neutral defaults + +Seriously considered — it is the smaller change and mirrors the +single-shared-category mechanism of +[`model-sample-absorption.md`](../accepted/model-sample-absorption.md) +(fields present everywhere, inert by default). Rejected as the primary +decision because the polarization coefficient is physically *zero* for +neutrons, not merely defaulted-off (sample absorption, by contrast, is +real for neutrons), so showing the knob on neutron experiments invites +meaningless tuning. Recorded as an Open Question because it avoids the +first-ever `radiation_probe` routing and the tag rename; if reviewers +judge the routing precedent too costly for two fields, this is the +fallback. + +### Add a new optics category now + +Rejected for now. There are only two fields and one concrete use case; +`experiment.instrument` is the natural category, and introducing an +optics abstraction before a second use case violates the +"don't introduce abstractions before a concrete second use case" +guidance in [`AGENTS.md`](../../../../AGENTS.md). + +## Deferred Work + +- Decide X-ray defaults once EasyDiffraction has an explicit incident + source model (`laboratory characteristic`, `synchrotron`, etc.). +- Confirm whether `_diffrn_radiation.polarisn_ratio` can safely map to + the EasyDiffraction polarization coefficient or whether it should stay + under the short `_instr.*` namespace permanently. +- File the separate CrysFML peak-shape window (`Wdt`) control issue + referenced in Decision 6. +- Extend the verification matrix after the scattering-table discrepancy + is resolved or explicitly gated. From c7e6ab1f6442e3077a4fc849f3961e8e65c4e969 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:04:08 +0200 Subject: [PATCH 02/16] Add X-ray CW polarization optics plan --- docs/dev/plans/xray-cw-polarization-optics.md | 301 ++++++++++++++++++ 1 file changed, 301 insertions(+) create mode 100644 docs/dev/plans/xray-cw-polarization-optics.md diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md new file mode 100644 index 000000000..0f40875da --- /dev/null +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -0,0 +1,301 @@ +# Implementation Plan: X-ray CW Polarization Optics + +This plan follows the conventions in [`AGENTS.md`](../../../AGENTS.md). +No deliberate exceptions to those instructions are taken. Per the +two-phase workflow, **Phase 1 is code + docs only** (no test creation +unless explicitly requested); tests and the five `pixi` verification +tasks run in **Phase 2**. When an AI agent executes this plan, **every +completed Phase 1 step must be staged with explicit paths and committed +locally before the next step** (atomic, single-purpose commits per +§Commits), and the agent stops at the Phase 1 review gate. + +## ADR + +Implements +[`docs/dev/adrs/suggestions/xray-cw-polarization-optics.md`](../adrs/suggestions/xray-cw-polarization-optics.md) +(Status: Proposed). The ADR is not yet promoted to `accepted/`; +`/draft-impl-1` commits it and promotes it to `accepted/` as part of the +implementation per §Change Discipline (move the file, flip Status to +Accepted, flip the existing **Experiment model** `index.md` row from +`Suggestion` to `Accepted` with an `accepted/…` link — see P1.1 — and +fix links). This plan uses the same slug as the ADR. + +Related accepted ADRs consulted: + +- [`model-sample-absorption.md`](../adrs/accepted/model-sample-absorption.md) + — the precedent for a CW-powder physics correction applied as a shared + pointwise multiplier after the engine returns (`absorption.apply`). +- [`immutable-experiment-type.md`](../adrs/accepted/immutable-experiment-type.md) + — `radiation_probe` is a creation-time axis, persisted in + `_expt_type.radiation_probe` and restored before the instrument is + rebuilt. +- [`iucr-cif-tag-alignment.md`](../adrs/accepted/iucr-cif-tag-alignment.md) + — experiment-tier fields keep short `_instr.*` names in the default + save. + +## Branch and PR + +- Flat-slug implementation branch off `develop`: + `xray-cw-polarization-optics` (created/checked out by `/draft-impl-1`). +- PR targets `develop`, not `master`. Do not push unless asked. + +## Dependencies + +No new runtime or dev dependencies. Cryspy already declares `k`/`cthm` +on its `Setup` item (defaults `k=0.0`, `cthm=0.91`), so binding is value +injection, not a backend extension. No `pyproject.toml` / `pixi.toml` / +`pixi.lock` changes are expected. + +## Decisions (already made in the ADR) + +1. The two values live on `experiment.instrument` (not a new optics + category, not a peak-shape model). +2. The CW **powder Bragg** instrument splits by `radiation_probe`: + `CwlPdInstrument` (tag `cwl-pd`) becomes `CwlPdNeutronInstrument` + (`cwl-pd-neutron`) and `CwlPdXrayInstrument` (`cwl-pd-xray`). The + X-ray class owns the optics fields; the neutron class does not. Single + crystal (`cwl-sc`), TOF (`tof-pd`/`tof-sc`), and total-scattering PDF + are unchanged. The identically named **data-range** `cwl-pd` tag is + out of scope and stays. Both new classes carry the full **Bragg-only** + compatibility tuple — `sample_form={POWDER}`, + `scattering_type={BRAGG}`, `beam_mode={CONSTANT_WAVELENGTH}`, plus + their `radiation_probe` — and `calculator_support={CRYSPY, CRYSFML}`. + The old class's `TOTAL` / `PDFFIT` metadata is dropped because + total-scattering PDF never routes through this instrument (the ADR + calls those entries unused aspirational metadata); empty + `Compatibility` axes mean "any", so the tuple must be stated in full + to avoid silently matching every sample form / beam mode. +3. Public names are physical and non-refinable (`NumericDescriptor`): + - `setup_polarization_coefficient` — dimensionless, default `0.0`, + range `[0, 1]` → cryspy `_setup_K` / FullProf `Rpolarz`. + - `setup_monochromator_twotheta` — degrees, default `0.0`, range + `[0, 180)` → backend `cthm = cos²(radians(2θm))` (default `0.0°` + ⇒ `cthm = 1.0`, "no monochromator"). +4. **cryspy binds natively**: emit `_setup_K` / `_setup_cthm` in the + generated CIF (and patch the cached dict like other instrument + scalars). cryspy applies the Lorentz-polarization factor internally; + the EasyDiffraction multiplier is **not** applied to cryspy output. +5. **crysfml binds via a post-pattern multiplier**, but only *after* + the ADR Decision 5 gate is satisfied: the implementer first verifies + (static source/CFL-grammar inspection) whether a native + polarization/monochromator CFL line exists. If one does, that native + line is emitted (and the ADR/plan updated); otherwise — the expected + outcome from the bundled CFL examples — `y *= lp_factor(two_theta, k, + cthm)` is applied after the engine returns, in a shared, unit-tested + helper. Two shared pieces: + `monochromator_cthm(angle)` (used by both adapters to produce the + backend `cthm`) and `lp_factor(two_theta, k, cthm)` (crysfml runtime + multiplier + the oracle that verifies cryspy's native output). +6. Defaults are neutron-neutral: with the coefficient at `0.0`, + `hh = 1` and results are unchanged until a user opts in. + +## Open questions + +1. **Verification reference data (P1.6).** The ADR Context references + `docs/docs/verification/pd-xray-pbso4-single-polarized-wdt48.py` + (does not exist yet) generated from + `pbsox_single_polarized_wdt48.pcr`. Producing a FullProf reference + needs FullProf output data committed under the verification data set. + **Question for the user:** is that reference data already available + to add, or should P1.6 instead demonstrate the new fields on the + existing `docs/docs/verification/pd-xray-pbso4.py` / + `docs/docs/tutorials/simulate-nacl-xray.py` (setting the coefficient + and showing the pattern responds) without a new FullProf page? The + plan defaults to the lighter demonstration if the data is not on hand. +2. **cryspy cached-dict keys (P1.4).** The CIF-build path is the primary + binding. Whether the cached working dict in + `_update_experiment_in_cryspy_dict` exposes `k`/`cthm` keys to patch + (mirroring `wavelength`) must be confirmed at implementation time; if + absent, rely on the CIF-build path plus cache rebuild on value change, + guarded like the existing `offset_sycos` key check. Because both + fields are non-refinable, the only scenario needing the patch is a + user changing the value after the dict is cached. + +## Concrete files likely to change + +Phase 1 (code + docs): + +- `src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py` + — split `CwlPdInstrument` into `CwlPdInstrumentBase` + + `CwlPdNeutronInstrument` + `CwlPdXrayInstrument`; declare the two + descriptors and their properties on the X-ray class. +- `.../instrument/factory.py` — replace the single powder-CW rule with + two `radiation_probe`-keyed rules. +- `.../instrument/__init__.py` — import/register the two new concrete + classes (drop the old `CwlPdInstrument` import). +- `src/easydiffraction/datablocks/experiment/item/bragg_pd.py` — pass + `radiation_probe=self.experiment_type.radiation_probe.value` into + `InstrumentFactory.default_tag(...)`. +- `src/easydiffraction/analysis/calculators/polarization.py` — **new** + shared helper: `monochromator_cthm`, `lp_factor`, `apply`. +- `src/easydiffraction/analysis/calculators/cryspy.py` — emit + `_setup_K` / `_setup_cthm` in `_cif_instrument_section`; optional + cached-dict patch in `_update_experiment_in_cryspy_dict`. +- `src/easydiffraction/analysis/calculators/crysfml.py` — apply + `polarization.apply(y, experiment)` on the return path (~line 170). +- A tutorial/verification source `.py` (P1.6, pending open question 1) + plus its regenerated notebook via `pixi run notebook-prepare`. +- ADR move to `accepted/` + `docs/dev/adrs/index.md` row. + +Phase 2 (tests + verification): + +- `tests/unit/.../instrument/test_factory.py`, + `tests/unit/.../analysis/calculators/test_support.py`, + `tests/unit/.../instrument/test_cwl.py`, and a new + `tests/unit/.../analysis/calculators/test_polarization.py`. +- `docs/dev/package-structure/{full,short}.md` (regenerated by + `pixi run fix`). + +## Implementation steps (Phase 1) + +- [ ] **P1.1 — Promote the ADR to `accepted/`.** Before any code edit, + `git mv docs/dev/adrs/suggestions/xray-cw-polarization-optics.md + docs/dev/adrs/accepted/`, set its `**Status:**` to `Accepted`, **flip + the existing** **Experiment model** row for this ADR in + `docs/dev/adrs/index.md` (currently `Suggestion`, ~line 45) **in + place** to `Accepted`, repointing its link from `suggestions/…` to + `accepted/xray-cw-polarization-optics.md` (do **not** add a second + row), update this plan's `## ADR` link to the `accepted/` path, and + fix every + remaining `suggestions/` reference found by + `git grep -n 'xray-cw-polarization-optics'`. `/draft-impl-1`'s Phase A + commits/cleans the ADR where it currently lives but does not invent + this promotion, so it is an explicit step here (per §Change + Discipline: an ADR-implementing change must not leave the ADR in + `suggestions/`). + Commit: `Promote xray-cw-polarization-optics ADR to accepted` + +- [ ] **P1.2 — Split the CW powder instrument by radiation probe.** + In `cwl.py`, extract a `CwlPdInstrumentBase(CwlInstrumentBase)` holding + the existing `calib_*` fields and properties. Add + `CwlPdNeutronInstrument` (tag `cwl-pd-neutron`) and + `CwlPdXrayInstrument` (tag `cwl-pd-xray`). Give **both** the full + Bragg-only `Compatibility`: + `sample_form=frozenset({SampleFormEnum.POWDER})`, + `scattering_type=frozenset({ScatteringTypeEnum.BRAGG})`, + `beam_mode=frozenset({BeamModeEnum.CONSTANT_WAVELENGTH})`, and + `radiation_probe=frozenset({RadiationProbeEnum.NEUTRON})` / + `{RadiationProbeEnum.XRAY})` respectively; and + `calculator_support=CalculatorSupport(calculators=frozenset( + {CalculatorEnum.CRYSPY, CalculatorEnum.CRYSFML}))` — **no `PDFFIT`, + no `TOTAL`**. On the X-ray class declare the two `NumericDescriptor`s + (with `DisplayHandler`, `AttributeSpec` defaults/validators, and + `TagSpec` exactly as ADR Decision 3) plus their getter/setter + properties (mirror the `setup_wavelength_2` property style). In + `factory.py` replace the `cwl-pd` rule with two rules keyed on + `radiation_probe`. In `bragg_pd.py` pass `radiation_probe=...` to + `default_tag(...)`. Update `instrument/__init__.py` registration. Leave + the data-range `cwl-pd` tag untouched. + Commit: `Split CW powder instrument by radiation probe` + +- [ ] **P1.3 — Add the shared Lorentz-polarization helper.** + New module `analysis/calculators/polarization.py` mirroring + `absorption.py`: `monochromator_cthm(twotheta_deg) -> float` + (`cos²(radians(twotheta))`), `lp_factor(two_theta, k, cthm) -> + np.ndarray` (`1 - k + k*cthm*cos²(two_theta)`), and `apply(y, + experiment) -> np.ndarray` that no-ops unless the instrument exposes + `setup_polarization_coefficient` with a non-zero value, otherwise + multiplies `y` by `lp_factor` over the experiment's 2θ grid. + Commit: `Add shared Lorentz-polarization helper` + +- [ ] **P1.4 — Bind cryspy natively.** + In `cryspy.py` `_cif_instrument_section`, within the existing powder + branch, when `hasattr(instrument, 'setup_polarization_coefficient')` + append `_setup_K ` and + `_setup_cthm ` + (import `monochromator_cthm` from P1.3). Do **not** apply + `polarization.apply` to cryspy output. If the cached working dict in + `_update_experiment_in_cryspy_dict` exposes the keys, patch them with a + presence guard (mirroring `offset_sycos`); otherwise document reliance + on the CIF-build path + cache rebuild (open question 2). + Commit: `Emit cryspy polarization setup for X-ray CW powder` + +- [ ] **P1.5 — Bind crysfml (verify native line first, then fall back).** + Satisfy the ADR Decision 5 gate explicitly: statically inspect the + active CrysFML CFL grammar / bundled examples for a native + polarization or monochromator line. **If one exists**, emit that native + line and update the ADR/plan to record it (or stop and ask if it + reopens scope). **If none exists** (the expected outcome), apply + `polarization.apply(y, experiment)` on the crysfml return path + alongside the existing `absorption_correction.apply(...)` + (~`crysfml.py:170`); no-op for neutron / zero coefficient. State the + inspection result in the commit body. + Commit: `Apply Lorentz-polarization envelope on crysfml CW powder` + +- [ ] **P1.6 — Demonstrate the new fields in docs.** + Per open question 1: either add the + `pd-xray-pbso4-single-polarized-wdt48.py` verification source (if the + FullProf reference data is available) or set + `setup_polarization_coefficient` / `setup_monochromator_twotheta` in an + existing X-ray CW source and show the pattern responds. Edit the `.py` + source only, then run `pixi run notebook-prepare` and commit the source + plus regenerated notebook. + Commit: `Document X-ray CW polarization optics fields` + +- [ ] **P1.7 — Phase 1 review gate.** No-code step. Mark complete and + commit the checklist update alone. + Commit: `Reach Phase 1 review gate` + +## Phase 2 — Verification + +Add/update tests, then run the five tasks. Capture logs with the +zsh-safe pattern where output is needed for analysis. + +Tests to add/update: + +- `test_factory.py` — `default_tag(..., radiation_probe=NEUTRON|XRAY)` + resolves to `cwl-pd-neutron` / `cwl-pd-xray`; `create()` for both tags; + remove `'cwl-pd'` instrument assertions. +- `test_support.py` — replace the `cwl-pd` calculator-support assertion + with the two new tags, asserting **only** the Bragg engines + (`{CRYSPY, CRYSFML}`) for `cwl-pd-neutron` / `cwl-pd-xray` (no + `PDFFIT`). +- `test_cwl.py` — the X-ray class exposes both descriptors with correct + defaults (`0.0`, `0.0`), range validation rejects out-of-range/wrong + type (`typeguard.TypeCheckError`), and the neutron class does **not** + expose them; CIF round-trip of `_instr.polarization_coefficient` / + `_instr.monochromator_twotheta`. +- new `test_polarization.py` — `monochromator_cthm(0) == 1.0`, the + `cos²` identity at a known angle, `lp_factor` reduces to `1` when + `k == 0`, and `apply` is a no-op for the neutron instrument. +- Do **not** modify the data-range `test_factory.py` `cwl-pd` cases. + +Commands: + +```bash +pixi run fix +pixi run check > /tmp/edi-check.log 2>&1; check_exit_code=$?; tail -n 200 /tmp/edi-check.log; exit $check_exit_code +pixi run unit-tests > /tmp/edi-unit.log 2>&1; unit_tests_exit_code=$?; tail -n 200 /tmp/edi-unit.log; exit $unit_tests_exit_code +pixi run integration-tests > /tmp/edi-integration.log 2>&1; integration_tests_exit_code=$?; tail -n 200 /tmp/edi-integration.log; exit $integration_tests_exit_code +pixi run script-tests > /tmp/edi-script.log 2>&1; script_tests_exit_code=$?; tail -n 200 /tmp/edi-script.log; exit $script_tests_exit_code +``` + +`pixi run fix` regenerates `docs/dev/package-structure/{full,short}.md`; +include them only in the `pixi run fix` commit. Leave generated +`docs/dev/benchmarking/*.csv` untracked unless asked. + +## Status checklist + +- [ ] P1.1 — Promote the ADR to `accepted/` and update `index.md` +- [ ] P1.2 — Split the CW powder instrument by radiation probe +- [ ] P1.3 — Add the shared Lorentz-polarization helper +- [ ] P1.4 — Bind cryspy natively +- [ ] P1.5 — Bind crysfml (verify native line first, then fall back) +- [ ] P1.6 — Demonstrate the new fields in docs +- [ ] P1.7 — Phase 1 review gate +- [ ] Phase 2 — tests added/updated and all five tasks pass + +## Suggested Pull Request + +**Title:** Match FullProf X-ray intensities with polarization and +monochromator settings + +**Description:** Constant-wavelength X-ray powder experiments now expose +two incident-optics controls — a polarization coefficient and a +pre-sample monochromator angle — so calculated intensities can match the +Lorentz-polarization correction used by FullProf and other Rietveld +codes. The settings appear only on X-ray powder instruments and are +applied by both the CrysPy and CrysFML engines; neutron experiments are +unaffected, and existing X-ray results stay the same until you set a +polarization value. This closes a known gap behind the residual +intensity mismatch seen in the PbSO₄ X-ray verification. From b7b07810c391b3132601ad2b56842c03093f7438 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:07:03 +0200 Subject: [PATCH 03/16] Promote xray-cw-polarization-optics ADR to accepted --- .../xray-cw-polarization-optics.md | 2 +- docs/dev/adrs/index.md | 1 + docs/dev/plans/xray-cw-polarization-optics.md | 33 ++++++------------- 3 files changed, 12 insertions(+), 24 deletions(-) rename docs/dev/adrs/{suggestions => accepted}/xray-cw-polarization-optics.md (99%) diff --git a/docs/dev/adrs/suggestions/xray-cw-polarization-optics.md b/docs/dev/adrs/accepted/xray-cw-polarization-optics.md similarity index 99% rename from docs/dev/adrs/suggestions/xray-cw-polarization-optics.md rename to docs/dev/adrs/accepted/xray-cw-polarization-optics.md index 6a84aefa2..5796c4b59 100644 --- a/docs/dev/adrs/suggestions/xray-cw-polarization-optics.md +++ b/docs/dev/adrs/accepted/xray-cw-polarization-optics.md @@ -2,7 +2,7 @@ ## Status -Proposed. +Accepted. ## Date diff --git a/docs/dev/adrs/index.md b/docs/dev/adrs/index.md index 8bd8a098a..e74e75f50 100644 --- a/docs/dev/adrs/index.md +++ b/docs/dev/adrs/index.md @@ -42,6 +42,7 @@ folders. | Experiment model | Accepted | Calculation Without Measured Data | Adds a writable `data_range` category so a structure-only experiment is calculable and plottable without loaded data. | [`calculation-without-measured-data.md`](accepted/calculation-without-measured-data.md) | | Experiment model | Accepted | Preferred-Orientation Category | Adds a per-phase March–Dollase preferred-orientation category for textured powder refinement on the CrysPy backend. | [`preferred-orientation-category.md`](accepted/preferred-orientation-category.md) | | Experiment model | Accepted | Model Sample Absorption (Debye–Scherrer, μR) | Switchable `absorption` category applying a calculator-independent cylindrical Hewat A(θ) envelope for powder samples. | [`model-sample-absorption.md`](accepted/model-sample-absorption.md) | +| Experiment model | Accepted | X-ray CW Polarization Optics | Adds discoverable X-ray CW powder instrument fields for FullProf/Cryspy Lorentz-polarization and monochromator optics. | [`xray-cw-polarization-optics.md`](accepted/xray-cw-polarization-optics.md) | | Factories | Accepted | Factory Contracts and Metadata | Standardizes factory construction, metadata, compatibility, and registration behavior. | [`factory-contracts.md`](accepted/factory-contracts.md) | | Naming | Accepted | Factory Tag Naming | Defines canonical factory tag style and standard abbreviations. | [`factory-tag-naming.md`](accepted/factory-tag-naming.md) | | Naming | Accepted | Downloadable Resource Naming | Replaces integer dataset/tutorial ids with stable descriptive slugs and moves presentation order into separate metadata. | [`resource-naming.md`](accepted/resource-naming.md) | diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index 0f40875da..e15a1a1e3 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -12,13 +12,9 @@ locally before the next step** (atomic, single-purpose commits per ## ADR Implements -[`docs/dev/adrs/suggestions/xray-cw-polarization-optics.md`](../adrs/suggestions/xray-cw-polarization-optics.md) -(Status: Proposed). The ADR is not yet promoted to `accepted/`; -`/draft-impl-1` commits it and promotes it to `accepted/` as part of the -implementation per §Change Discipline (move the file, flip Status to -Accepted, flip the existing **Experiment model** `index.md` row from -`Suggestion` to `Accepted` with an `accepted/…` link — see P1.1 — and -fix links). This plan uses the same slug as the ADR. +[`docs/dev/adrs/accepted/xray-cw-polarization-optics.md`](../adrs/accepted/xray-cw-polarization-optics.md) +(Status: Accepted). The ADR was promoted to `accepted/` during P1.1 per +§Change Discipline. This plan uses the same slug as the ADR. Related accepted ADRs consulted: @@ -148,21 +144,12 @@ Phase 2 (tests + verification): ## Implementation steps (Phase 1) -- [ ] **P1.1 — Promote the ADR to `accepted/`.** Before any code edit, - `git mv docs/dev/adrs/suggestions/xray-cw-polarization-optics.md - docs/dev/adrs/accepted/`, set its `**Status:**` to `Accepted`, **flip - the existing** **Experiment model** row for this ADR in - `docs/dev/adrs/index.md` (currently `Suggestion`, ~line 45) **in - place** to `Accepted`, repointing its link from `suggestions/…` to - `accepted/xray-cw-polarization-optics.md` (do **not** add a second - row), update this plan's `## ADR` link to the `accepted/` path, and - fix every - remaining `suggestions/` reference found by - `git grep -n 'xray-cw-polarization-optics'`. `/draft-impl-1`'s Phase A - commits/cleans the ADR where it currently lives but does not invent - this promotion, so it is an explicit step here (per §Change - Discipline: an ADR-implementing change must not leave the ADR in - `suggestions/`). +- [x] **P1.1 — Promote the ADR to `accepted/`.** The ADR was moved into + `docs/dev/adrs/accepted/`, its status was set to `Accepted`, the + existing **Experiment model** row in `docs/dev/adrs/index.md` was + repointed to `accepted/xray-cw-polarization-optics.md`, this plan's + `## ADR` link was updated, and remaining stale path references were + cleared. Commit: `Promote xray-cw-polarization-optics ADR to accepted` - [ ] **P1.2 — Split the CW powder instrument by radiation probe.** @@ -276,7 +263,7 @@ include them only in the `pixi run fix` commit. Leave generated ## Status checklist -- [ ] P1.1 — Promote the ADR to `accepted/` and update `index.md` +- [x] P1.1 — Promote the ADR to `accepted/` and update `index.md` - [ ] P1.2 — Split the CW powder instrument by radiation probe - [ ] P1.3 — Add the shared Lorentz-polarization helper - [ ] P1.4 — Bind cryspy natively From 71f60b2737a59ce23277982906542bedfd1709fc Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:09:46 +0200 Subject: [PATCH 04/16] Split CW powder instrument by radiation probe --- docs/dev/plans/xray-cw-polarization-optics.md | 4 +- .../categories/instrument/__init__.py | 5 +- .../experiment/categories/instrument/cwl.py | 149 +++++++++++++++--- .../categories/instrument/factory.py | 12 +- .../datablocks/experiment/item/bragg_pd.py | 1 + 5 files changed, 147 insertions(+), 24 deletions(-) diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index e15a1a1e3..ab3697cd0 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -152,7 +152,7 @@ Phase 2 (tests + verification): cleared. Commit: `Promote xray-cw-polarization-optics ADR to accepted` -- [ ] **P1.2 — Split the CW powder instrument by radiation probe.** +- [x] **P1.2 — Split the CW powder instrument by radiation probe.** In `cwl.py`, extract a `CwlPdInstrumentBase(CwlInstrumentBase)` holding the existing `calib_*` fields and properties. Add `CwlPdNeutronInstrument` (tag `cwl-pd-neutron`) and @@ -264,7 +264,7 @@ include them only in the `pixi run fix` commit. Leave generated ## Status checklist - [x] P1.1 — Promote the ADR to `accepted/` and update `index.md` -- [ ] P1.2 — Split the CW powder instrument by radiation probe +- [x] P1.2 — Split the CW powder instrument by radiation probe - [ ] P1.3 — Add the shared Lorentz-polarization helper - [ ] P1.4 — Bind cryspy natively - [ ] P1.5 — Bind crysfml (verify native line first, then fall back) diff --git a/src/easydiffraction/datablocks/experiment/categories/instrument/__init__.py b/src/easydiffraction/datablocks/experiment/categories/instrument/__init__.py index 713847025..aaf41f525 100644 --- a/src/easydiffraction/datablocks/experiment/categories/instrument/__init__.py +++ b/src/easydiffraction/datablocks/experiment/categories/instrument/__init__.py @@ -2,7 +2,10 @@ # SPDX-License-Identifier: BSD-3-Clause """Instrument categories for CWL and TOF powder and single crystal.""" -from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdInstrument +from __future__ import annotations + +from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument +from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdXrayInstrument from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlScInstrument from easydiffraction.datablocks.experiment.categories.instrument.tof import TofPdInstrument from easydiffraction.datablocks.experiment.categories.instrument.tof import TofScInstrument diff --git a/src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py b/src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py index bafb8d96f..e3ccbd58e 100644 --- a/src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py +++ b/src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py @@ -2,6 +2,8 @@ # SPDX-License-Identifier: BSD-3-Clause """Constant-wavelength powder and single-crystal instruments.""" +from __future__ import annotations + from easydiffraction.core.display_handler import DisplayHandler from easydiffraction.core.metadata import CalculatorSupport from easydiffraction.core.metadata import Compatibility @@ -14,6 +16,7 @@ from easydiffraction.datablocks.experiment.categories.instrument.factory import InstrumentFactory from easydiffraction.datablocks.experiment.item.enums import BeamModeEnum from easydiffraction.datablocks.experiment.item.enums import CalculatorEnum +from easydiffraction.datablocks.experiment.item.enums import RadiationProbeEnum from easydiffraction.datablocks.experiment.item.enums import SampleFormEnum from easydiffraction.datablocks.experiment.item.enums import ScatteringTypeEnum from easydiffraction.io.cif.handler import TagSpec @@ -164,26 +167,8 @@ def __init__(self) -> None: super().__init__() -@InstrumentFactory.register -class CwlPdInstrument(CwlInstrumentBase): - """CW powder diffractometer.""" - - type_info = TypeInfo( - tag='cwl-pd', - description='CW powder diffractometer', - ) - compatibility = Compatibility( - scattering_type=frozenset({ScatteringTypeEnum.BRAGG, ScatteringTypeEnum.TOTAL}), - beam_mode=frozenset({BeamModeEnum.CONSTANT_WAVELENGTH}), - sample_form=frozenset({SampleFormEnum.POWDER}), - ) - calculator_support = CalculatorSupport( - calculators=frozenset({ - CalculatorEnum.CRYSPY, - CalculatorEnum.CRYSFML, - CalculatorEnum.PDFFIT, - }), - ) +class CwlPdInstrumentBase(CwlInstrumentBase): + """Base class for CW powder diffractometers.""" def __init__(self) -> None: """Initialize the CW powder diffractometer.""" @@ -293,3 +278,127 @@ def calib_sample_transparency(self) -> Parameter: def calib_sample_transparency(self, value: float) -> None: """Set the sample-transparency correction (deg).""" self._calib_sample_transparency.value = value + + +@InstrumentFactory.register +class CwlPdNeutronInstrument(CwlPdInstrumentBase): + """CW neutron powder diffractometer.""" + + type_info = TypeInfo( + tag='cwl-pd-neutron', + description='CW neutron powder diffractometer', + ) + compatibility = Compatibility( + scattering_type=frozenset({ScatteringTypeEnum.BRAGG}), + beam_mode=frozenset({BeamModeEnum.CONSTANT_WAVELENGTH}), + sample_form=frozenset({SampleFormEnum.POWDER}), + radiation_probe=frozenset({RadiationProbeEnum.NEUTRON}), + ) + calculator_support = CalculatorSupport( + calculators=frozenset({ + CalculatorEnum.CRYSPY, + CalculatorEnum.CRYSFML, + }), + ) + + def __init__(self) -> None: + """Initialize the CW neutron powder diffractometer.""" + super().__init__() + + +@InstrumentFactory.register +class CwlPdXrayInstrument(CwlPdInstrumentBase): + """CW X-ray powder diffractometer.""" + + type_info = TypeInfo( + tag='cwl-pd-xray', + description='CW X-ray powder diffractometer', + ) + compatibility = Compatibility( + scattering_type=frozenset({ScatteringTypeEnum.BRAGG}), + beam_mode=frozenset({BeamModeEnum.CONSTANT_WAVELENGTH}), + sample_form=frozenset({SampleFormEnum.POWDER}), + radiation_probe=frozenset({RadiationProbeEnum.XRAY}), + ) + calculator_support = CalculatorSupport( + calculators=frozenset({ + CalculatorEnum.CRYSPY, + CalculatorEnum.CRYSFML, + }), + ) + + def __init__(self) -> None: + """Initialize the CW X-ray powder diffractometer.""" + super().__init__() + + self._setup_polarization_coefficient: NumericDescriptor = NumericDescriptor( + name='polarization_coefficient', + description='CW Lorentz-polarization coefficient', + units='', + display_handler=DisplayHandler( + display_name='Polarization coefficient', + display_units='', + latex_name='Polarization coefficient', + latex_units='', + ), + value_spec=AttributeSpec( + default=0.0, + validator=RangeValidator(ge=0.0, le=1.0), + ), + tags=TagSpec( + edi_names=['_instrument.setup_polarization_coefficient'], + cif_names=['_instr.polarization_coefficient'], + ), + ) + + self._setup_monochromator_twotheta: NumericDescriptor = NumericDescriptor( + name='monochromator_twotheta', + description='Pre-specimen monochromator 2theta angle', + units='degrees', + display_handler=DisplayHandler( + display_name='Monochromator 2θ', + display_units='deg', + latex_name=r'Monochromator $2\theta$', + latex_units=r'\mathrm{deg}', + ), + value_spec=AttributeSpec( + default=0.0, + validator=RangeValidator(ge=0.0, lt=180.0), + ), + tags=TagSpec( + edi_names=['_instrument.setup_monochromator_twotheta'], + cif_names=['_instr.monochromator_twotheta'], + ), + ) + + @property + def setup_polarization_coefficient(self) -> NumericDescriptor: + """ + CW Lorentz-polarization coefficient. + + Reading returns the underlying ``NumericDescriptor``; assigning + a number updates its value. Default ``0.0`` disables the + polarization correction. + """ + return self._setup_polarization_coefficient + + @setup_polarization_coefficient.setter + def setup_polarization_coefficient(self, value: float) -> None: + """Set the CW Lorentz-polarization coefficient.""" + self._setup_polarization_coefficient.value = value + + @property + def setup_monochromator_twotheta(self) -> NumericDescriptor: + """ + Pre-specimen monochromator 2theta angle (deg). + + Reading returns the underlying ``NumericDescriptor``; assigning + a number updates its value. Default ``0.0`` means no + monochromator. + """ + return self._setup_monochromator_twotheta + + @setup_monochromator_twotheta.setter + def setup_monochromator_twotheta(self, value: float) -> None: + """Set the pre-specimen monochromator 2theta angle (deg).""" + self._setup_monochromator_twotheta.value = value diff --git a/src/easydiffraction/datablocks/experiment/categories/instrument/factory.py b/src/easydiffraction/datablocks/experiment/categories/instrument/factory.py index 5700e844e..cda5a7a70 100644 --- a/src/easydiffraction/datablocks/experiment/categories/instrument/factory.py +++ b/src/easydiffraction/datablocks/experiment/categories/instrument/factory.py @@ -8,7 +8,9 @@ from easydiffraction.core.factory import FactoryBase from easydiffraction.datablocks.experiment.item.enums import BeamModeEnum +from easydiffraction.datablocks.experiment.item.enums import RadiationProbeEnum from easydiffraction.datablocks.experiment.item.enums import SampleFormEnum +from easydiffraction.datablocks.experiment.item.enums import ScatteringTypeEnum class InstrumentFactory(FactoryBase): @@ -18,7 +20,15 @@ class InstrumentFactory(FactoryBase): frozenset({ ('beam_mode', BeamModeEnum.CONSTANT_WAVELENGTH), ('sample_form', SampleFormEnum.POWDER), - }): 'cwl-pd', + ('scattering_type', ScatteringTypeEnum.BRAGG), + ('radiation_probe', RadiationProbeEnum.NEUTRON), + }): 'cwl-pd-neutron', + frozenset({ + ('beam_mode', BeamModeEnum.CONSTANT_WAVELENGTH), + ('sample_form', SampleFormEnum.POWDER), + ('scattering_type', ScatteringTypeEnum.BRAGG), + ('radiation_probe', RadiationProbeEnum.XRAY), + }): 'cwl-pd-xray', frozenset({ ('beam_mode', BeamModeEnum.CONSTANT_WAVELENGTH), ('sample_form', SampleFormEnum.SINGLE_CRYSTAL), diff --git a/src/easydiffraction/datablocks/experiment/item/bragg_pd.py b/src/easydiffraction/datablocks/experiment/item/bragg_pd.py index 2948f63ea..e47087b7f 100644 --- a/src/easydiffraction/datablocks/experiment/item/bragg_pd.py +++ b/src/easydiffraction/datablocks/experiment/item/bragg_pd.py @@ -62,6 +62,7 @@ def __init__( scattering_type=self.experiment_type.scattering_type.value, beam_mode=self.experiment_type.beam_mode.value, sample_form=self.experiment_type.sample_form.value, + radiation_probe=self.experiment_type.radiation_probe.value, ) self._instrument = InstrumentFactory.create(self._instrument_type) self._background = BackgroundFactory.create(BackgroundFactory.default_tag()) From 69d6c118d9d9f8de1231a352b7e5e0dc23c8e0e4 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:11:06 +0200 Subject: [PATCH 05/16] Add shared Lorentz-polarization helper --- docs/dev/plans/xray-cw-polarization-optics.md | 4 +- .../analysis/calculators/polarization.py | 97 +++++++++++++++++++ 2 files changed, 99 insertions(+), 2 deletions(-) create mode 100644 src/easydiffraction/analysis/calculators/polarization.py diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index ab3697cd0..a2159f253 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -175,7 +175,7 @@ Phase 2 (tests + verification): the data-range `cwl-pd` tag untouched. Commit: `Split CW powder instrument by radiation probe` -- [ ] **P1.3 — Add the shared Lorentz-polarization helper.** +- [x] **P1.3 — Add the shared Lorentz-polarization helper.** New module `analysis/calculators/polarization.py` mirroring `absorption.py`: `monochromator_cthm(twotheta_deg) -> float` (`cos²(radians(twotheta))`), `lp_factor(two_theta, k, cthm) -> @@ -265,7 +265,7 @@ include them only in the `pixi run fix` commit. Leave generated - [x] P1.1 — Promote the ADR to `accepted/` and update `index.md` - [x] P1.2 — Split the CW powder instrument by radiation probe -- [ ] P1.3 — Add the shared Lorentz-polarization helper +- [x] P1.3 — Add the shared Lorentz-polarization helper - [ ] P1.4 — Bind cryspy natively - [ ] P1.5 — Bind crysfml (verify native line first, then fall back) - [ ] P1.6 — Demonstrate the new fields in docs diff --git a/src/easydiffraction/analysis/calculators/polarization.py b/src/easydiffraction/analysis/calculators/polarization.py new file mode 100644 index 000000000..b9cb79840 --- /dev/null +++ b/src/easydiffraction/analysis/calculators/polarization.py @@ -0,0 +1,97 @@ +# SPDX-FileCopyrightText: 2026 EasyScience contributors +# SPDX-License-Identifier: BSD-3-Clause +""" +Calculator-independent CW Lorentz-polarization correction factor. + +The public instrument stores the physical monochromator angle in +degrees. Backends that expose the FullProf/Cryspy form consume +``cthm = cos^2(2theta_m)``, while backends without a native field use +the same value in the pointwise multiplier here. +""" + +from __future__ import annotations + +import numpy as np + + +def monochromator_cthm(two_theta_m: float) -> float: + """ + Return the backend monochromator ``cthm`` value. + + Parameters + ---------- + two_theta_m : float + Pre-specimen monochromator 2-theta angle in degrees. + + Returns + ------- + float + ``cos^2(two_theta_m)`` with the angle interpreted in degrees. + """ + return float(np.cos(np.radians(two_theta_m)) ** 2) + + +def lp_factor( + two_theta: np.ndarray, + polarization_coefficient: float, + cthm: float, +) -> np.ndarray: + """ + Return the CW Lorentz-polarization multiplier. + + Parameters + ---------- + two_theta : np.ndarray + Scattering angle 2-theta grid in degrees. + polarization_coefficient : float + Dimensionless polarization coefficient, ``0`` to ``1``. + cthm : float + Monochromator factor ``cos^2(two_theta_m)``. + + Returns + ------- + np.ndarray + Multiplicative Lorentz-polarization factor. + """ + two_theta = np.asarray(two_theta, dtype=float) + cos2 = np.cos(np.radians(two_theta)) ** 2 + return 1.0 - polarization_coefficient + polarization_coefficient * cthm * cos2 + + +def apply(y: object, experiment: object) -> object: + """ + Apply the CW Lorentz-polarization factor to calculated intensities. + + The pattern is returned unchanged unless the experiment has an + X-ray CW powder instrument with a nonzero polarization coefficient. + Empty arrays and backend no-data paths whose shape does not match + the experiment 2-theta grid are also left unchanged. + + Parameters + ---------- + y : object + Calculated intensities (NumPy array or list). + experiment : object + Experiment providing ``instrument`` and ``data.x``. + + Returns + ------- + object + Corrected intensities, or ``y`` unchanged when no correction + applies. + """ + instrument = getattr(experiment, 'instrument', None) + if not hasattr(instrument, 'setup_polarization_coefficient'): + return y + + coefficient = instrument.setup_polarization_coefficient.value + if coefficient == 0.0: + return y + + y_values = np.asarray(y, dtype=float) + two_theta = np.asarray(experiment.data.x, dtype=float) + if y_values.size == 0 or y_values.shape != two_theta.shape: + return y + + cthm = monochromator_cthm(instrument.setup_monochromator_twotheta.value) + return y_values * lp_factor(two_theta, coefficient, cthm) From 19ce3f5141eb695c7f1211bb41d37cd16c2e4a76 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:13:59 +0200 Subject: [PATCH 06/16] Emit cryspy polarization setup for X-ray CW powder --- docs/dev/plans/xray-cw-polarization-optics.md | 17 ++-- .../analysis/calculators/cryspy.py | 85 ++++++++++++++++++- 2 files changed, 88 insertions(+), 14 deletions(-) diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index a2159f253..7871693ea 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -98,14 +98,11 @@ injection, not a backend extension. No `pyproject.toml` / `pixi.toml` / `docs/docs/tutorials/simulate-nacl-xray.py` (setting the coefficient and showing the pattern responds) without a new FullProf page? The plan defaults to the lighter demonstration if the data is not on hand. -2. **cryspy cached-dict keys (P1.4).** The CIF-build path is the primary - binding. Whether the cached working dict in - `_update_experiment_in_cryspy_dict` exposes `k`/`cthm` keys to patch - (mirroring `wavelength`) must be confirmed at implementation time; if - absent, rely on the CIF-build path plus cache rebuild on value change, - guarded like the existing `offset_sycos` key check. Because both - fields are non-refinable, the only scenario needing the patch is a - user changing the value after the dict is cached. +2. **cryspy cached-dict keys (P1.4).** Resolved in P1.4: the CIF-build + path emits `_setup_K` and `_setup_cthm`, the cached working dict is + patched when `k`/`cthm` keys are exposed, and polarization settings + are part of the cache-invalidation key so releases without those + patch keys rebuild from CIF on value changes. ## Concrete files likely to change @@ -185,7 +182,7 @@ Phase 2 (tests + verification): multiplies `y` by `lp_factor` over the experiment's 2θ grid. Commit: `Add shared Lorentz-polarization helper` -- [ ] **P1.4 — Bind cryspy natively.** +- [x] **P1.4 — Bind cryspy natively.** In `cryspy.py` `_cif_instrument_section`, within the existing powder branch, when `hasattr(instrument, 'setup_polarization_coefficient')` append `_setup_K ` and @@ -266,7 +263,7 @@ include them only in the `pixi run fix` commit. Leave generated - [x] P1.1 — Promote the ADR to `accepted/` and update `index.md` - [x] P1.2 — Split the CW powder instrument by radiation probe - [x] P1.3 — Add the shared Lorentz-polarization helper -- [ ] P1.4 — Bind cryspy natively +- [x] P1.4 — Bind cryspy natively - [ ] P1.5 — Bind crysfml (verify native line first, then fall back) - [ ] P1.6 — Demonstrate the new fields in docs - [ ] P1.7 — Phase 1 review gate diff --git a/src/easydiffraction/analysis/calculators/cryspy.py b/src/easydiffraction/analysis/calculators/cryspy.py index 20f11ad13..5a0b1e67b 100644 --- a/src/easydiffraction/analysis/calculators/cryspy.py +++ b/src/easydiffraction/analysis/calculators/cryspy.py @@ -13,6 +13,7 @@ import numpy as np from easydiffraction.analysis.calculators import absorption as absorption_correction +from easydiffraction.analysis.calculators import polarization as polarization_correction from easydiffraction.analysis.calculators.base import CalculatorBase from easydiffraction.analysis.calculators.base import PowderReflnRecord from easydiffraction.analysis.calculators.factory import CalculatorFactory @@ -72,6 +73,7 @@ def __init__(self) -> None: self._cached_peak_types: dict[str, str] = {} self._cached_adp_types: dict[str, tuple[str, ...]] = {} self._cached_pref_orient: dict[str, tuple] = {} + self._cached_polarization_settings: dict[str, tuple[float, float] | None] = {} self._last_powder_phase_blocks: dict[str, dict[str, Any] | None] = {} def _invalidate_stale_cache( @@ -83,10 +85,10 @@ def _invalidate_stale_cache( """ Drop cached dict when experiment or structure config changed. - Checks the peak profile type, the per-atom ADP types, and the - preferred-orientation row identities. When any changes the - cached dictionary is stale and must be rebuilt from a fresh - cryspy object. + Checks the peak profile type, per-atom ADP types, + preferred-orientation row identities, and polarization optics. + When any changes the cached dictionary is stale and must be + rebuilt from a fresh cryspy object. """ if 'peak' in type(experiment)._public_attrs(): current_type = experiment.peak.type_info.tag @@ -119,6 +121,11 @@ def _invalidate_stale_cache( self._cryspy_dicts.pop(combined_name, None) self._cached_pref_orient[combined_name] = current_pref_orient + current_polarization = _polarization_settings(experiment) + if self._cached_polarization_settings.get(combined_name) != current_polarization: + self._cryspy_dicts.pop(combined_name, None) + self._cached_polarization_settings[combined_name] = current_polarization + if structure is not None: current_adp = tuple(atom.adp_type.value for atom in structure.atom_sites) if self._cached_adp_types.get(combined_name) != current_adp: @@ -711,6 +718,10 @@ def _update_experiment_in_cryspy_dict( cryspy_expt_dict['offset_sysin'][0] = ( experiment.instrument.calib_sample_transparency.value ) + _update_polarization_in_cryspy_dict( + cryspy_expt_dict, + experiment.instrument, + ) # Peak cryspy_resolution = cryspy_expt_dict['resolution_parameters'] @@ -1158,6 +1169,72 @@ def _cif_instrument_section( if attr_obj is not None: cif_lines.append(f'{engine_key_name} {attr_obj.value}') + _cif_polarization_section(cif_lines, instrument) + + +def _cif_polarization_section( + cif_lines: list[str], + instrument: object, +) -> None: + """Append native Cryspy polarization setup lines when available.""" + settings = _polarization_settings_from_instrument(instrument) + if settings is None: + return + coefficient, monochromator_twotheta = settings + cthm = polarization_correction.monochromator_cthm(monochromator_twotheta) + cif_lines.append(f'_setup_K {coefficient}') + cif_lines.append(f'_setup_cthm {cthm}') + + +def _polarization_settings(experiment: object) -> tuple[float, float] | None: + """Return polarization settings from an experiment, if present.""" + instrument = getattr(experiment, 'instrument', None) + return _polarization_settings_from_instrument(instrument) + + +def _polarization_settings_from_instrument( + instrument: object | None, +) -> tuple[float, float] | None: + """Return polarization settings from an instrument, if present.""" + if not hasattr(instrument, 'setup_polarization_coefficient'): + return None + return ( + instrument.setup_polarization_coefficient.value, + instrument.setup_monochromator_twotheta.value, + ) + + +def _update_polarization_in_cryspy_dict( + cryspy_expt_dict: dict[str, Any], + instrument: object, +) -> None: + """Patch native Cryspy polarization setup keys when exposed.""" + settings = _polarization_settings_from_instrument(instrument) + if settings is None: + return + coefficient, monochromator_twotheta = settings + if 'k' in cryspy_expt_dict: + _set_cryspy_scalar(cryspy_expt_dict, 'k', coefficient) + if 'cthm' in cryspy_expt_dict: + _set_cryspy_scalar( + cryspy_expt_dict, + 'cthm', + polarization_correction.monochromator_cthm(monochromator_twotheta), + ) + + +def _set_cryspy_scalar( + cryspy_expt_dict: dict[str, Any], + key: str, + value: float, +) -> None: + """Set a Cryspy scalar stored either directly or in a 1-item array.""" + target = cryspy_expt_dict[key] + if isinstance(target, (np.ndarray, list)): + target[0] = value + return + cryspy_expt_dict[key] = value + def _update_tof_peak_in_cryspy_dict( cryspy_expt_dict: dict[str, Any], From abee751d463ae456d63538196fc03cfc41070ca1 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:16:11 +0200 Subject: [PATCH 07/16] Apply Lorentz-polarization envelope on crysfml CW powder Static inspection found lower-level CrysFML Lorentz routines that accept cthm/rkk, but the Python cw_powder_pattern_from_dict wrapper reads no dictionary keys for those values. Use the shared post-pattern multiplier for the wrapper path. --- docs/dev/plans/xray-cw-polarization-optics.md | 8 ++++++-- src/easydiffraction/analysis/calculators/crysfml.py | 5 ++++- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index 7871693ea..87e8a2e0f 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -82,6 +82,10 @@ injection, not a backend extension. No `pyproject.toml` / `pixi.toml` / `monochromator_cthm(angle)` (used by both adapters to produce the backend `cthm`) and `lp_factor(two_theta, k, cthm)` (crysfml runtime multiplier + the oracle that verifies cryspy's native output). + P1.5 inspection found lower-level CrysFML Lorentz routines with + optional `cthm`/`rkk` arguments, but the Python + `cw_powder_pattern_from_dict` wrapper reads no dictionary keys for + them, so the planned fallback multiplier is used. 6. Defaults are neutron-neutral: with the coefficient at `0.0`, `hh = 1` and results are unchanged until a user opts in. @@ -194,7 +198,7 @@ Phase 2 (tests + verification): on the CIF-build path + cache rebuild (open question 2). Commit: `Emit cryspy polarization setup for X-ray CW powder` -- [ ] **P1.5 — Bind crysfml (verify native line first, then fall back).** +- [x] **P1.5 — Bind crysfml (verify native line first, then fall back).** Satisfy the ADR Decision 5 gate explicitly: statically inspect the active CrysFML CFL grammar / bundled examples for a native polarization or monochromator line. **If one exists**, emit that native @@ -264,7 +268,7 @@ include them only in the `pixi run fix` commit. Leave generated - [x] P1.2 — Split the CW powder instrument by radiation probe - [x] P1.3 — Add the shared Lorentz-polarization helper - [x] P1.4 — Bind cryspy natively -- [ ] P1.5 — Bind crysfml (verify native line first, then fall back) +- [x] P1.5 — Bind crysfml (verify native line first, then fall back) - [ ] P1.6 — Demonstrate the new fields in docs - [ ] P1.7 — Phase 1 review gate - [ ] Phase 2 — tests added/updated and all five tasks pass diff --git a/src/easydiffraction/analysis/calculators/crysfml.py b/src/easydiffraction/analysis/calculators/crysfml.py index 67414f4af..c51003f41 100644 --- a/src/easydiffraction/analysis/calculators/crysfml.py +++ b/src/easydiffraction/analysis/calculators/crysfml.py @@ -11,6 +11,7 @@ import numpy as np from easydiffraction.analysis.calculators import absorption as absorption_correction +from easydiffraction.analysis.calculators import polarization as polarization_correction from easydiffraction.analysis.calculators.base import CalculatorBase from easydiffraction.analysis.calculators.factory import CalculatorFactory from easydiffraction.core.metadata import TypeInfo @@ -167,7 +168,9 @@ def calculate_pattern( except KeyError: log.warning('[CrysfmlCalculator] No calculated data') y = [] - return np.asarray(absorption_correction.apply(y, experiment)) + y = absorption_correction.apply(y, experiment) + y = polarization_correction.apply(y, experiment) + return np.asarray(y) def _calculate_adjusted_pattern( self, From 8f4ee82de8399e0dabd8aafbd297fde0421a66e3 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:20:04 +0200 Subject: [PATCH 08/16] Document X-ray CW polarization optics fields --- docs/dev/plans/xray-cw-polarization-optics.md | 20 ++---- docs/docs/tutorials/simulate-nacl-xray.ipynb | 64 +++++++++++++++++-- docs/docs/tutorials/simulate-nacl-xray.py | 23 +++++++ 3 files changed, 90 insertions(+), 17 deletions(-) diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index 87e8a2e0f..fceff781a 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -91,17 +91,11 @@ injection, not a backend extension. No `pyproject.toml` / `pixi.toml` / ## Open questions -1. **Verification reference data (P1.6).** The ADR Context references - `docs/docs/verification/pd-xray-pbso4-single-polarized-wdt48.py` - (does not exist yet) generated from - `pbsox_single_polarized_wdt48.pcr`. Producing a FullProf reference - needs FullProf output data committed under the verification data set. - **Question for the user:** is that reference data already available - to add, or should P1.6 instead demonstrate the new fields on the - existing `docs/docs/verification/pd-xray-pbso4.py` / - `docs/docs/tutorials/simulate-nacl-xray.py` (setting the coefficient - and showing the pattern responds) without a new FullProf page? The - plan defaults to the lighter demonstration if the data is not on hand. +1. **Verification reference data (P1.6).** Resolved in P1.6: no new + FullProf polarized reference data was present, so the implementation + used the plan's lighter demonstration path in + `docs/docs/tutorials/simulate-nacl-xray.py` and regenerated the + matching notebook. 2. **cryspy cached-dict keys (P1.4).** Resolved in P1.4: the CIF-build path emits `_setup_K` and `_setup_cthm`, the cached working dict is patched when `k`/`cthm` keys are exposed, and polarization settings @@ -210,7 +204,7 @@ Phase 2 (tests + verification): inspection result in the commit body. Commit: `Apply Lorentz-polarization envelope on crysfml CW powder` -- [ ] **P1.6 — Demonstrate the new fields in docs.** +- [x] **P1.6 — Demonstrate the new fields in docs.** Per open question 1: either add the `pd-xray-pbso4-single-polarized-wdt48.py` verification source (if the FullProf reference data is available) or set @@ -269,7 +263,7 @@ include them only in the `pixi run fix` commit. Leave generated - [x] P1.3 — Add the shared Lorentz-polarization helper - [x] P1.4 — Bind cryspy natively - [x] P1.5 — Bind crysfml (verify native line first, then fall back) -- [ ] P1.6 — Demonstrate the new fields in docs +- [x] P1.6 — Demonstrate the new fields in docs - [ ] P1.7 — Phase 1 review gate - [ ] Phase 2 — tests added/updated and all five tasks pass diff --git a/docs/docs/tutorials/simulate-nacl-xray.ipynb b/docs/docs/tutorials/simulate-nacl-xray.ipynb index 5c82b1c71..6ae11ab4c 100644 --- a/docs/docs/tutorials/simulate-nacl-xray.ipynb +++ b/docs/docs/tutorials/simulate-nacl-xray.ipynb @@ -51,7 +51,8 @@ "metadata": {}, "outputs": [], "source": [ - "import easydiffraction as edi" + "import easydiffraction as edi\n", + "import numpy as np" ] }, { @@ -264,6 +265,61 @@ "cell_type": "markdown", "id": "23", "metadata": {}, + "source": [ + "### Set X-ray Polarization Optics\n", + "\n", + "The polarization coefficient defaults to `0.0`. Setting it to a\n", + "nonzero value includes the Lorentz-polarization optics in the\n", + "calculated X-ray intensities." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "24", + "metadata": {}, + "outputs": [], + "source": [ + "project.analysis.calculate()\n", + "unpolarized_intensity = np.asarray(experiment.data.intensity_calc).copy()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "25", + "metadata": {}, + "outputs": [], + "source": [ + "experiment.instrument.setup_polarization_coefficient = 0.5\n", + "experiment.instrument.setup_monochromator_twotheta = 26.5650511771" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "26", + "metadata": {}, + "outputs": [], + "source": [ + "project.analysis.calculate()\n", + "polarized_intensity = np.asarray(experiment.data.intensity_calc).copy()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "27", + "metadata": {}, + "outputs": [], + "source": [ + "print('max intensity change:', np.max(np.abs(polarized_intensity - unpolarized_intensity)))" + ] + }, + { + "cell_type": "markdown", + "id": "28", + "metadata": {}, "source": [ "## 🚀 Perform Calculation\n", "\n", @@ -273,7 +329,7 @@ { "cell_type": "code", "execution_count": null, - "id": "24", + "id": "29", "metadata": {}, "outputs": [], "source": [ @@ -282,7 +338,7 @@ }, { "cell_type": "markdown", - "id": "25", + "id": "30", "metadata": {}, "source": [ "## 💾 Save Project" @@ -291,7 +347,7 @@ { "cell_type": "code", "execution_count": null, - "id": "26", + "id": "31", "metadata": {}, "outputs": [], "source": [ diff --git a/docs/docs/tutorials/simulate-nacl-xray.py b/docs/docs/tutorials/simulate-nacl-xray.py index 65f1065d3..dd880d1b1 100644 --- a/docs/docs/tutorials/simulate-nacl-xray.py +++ b/docs/docs/tutorials/simulate-nacl-xray.py @@ -15,6 +15,7 @@ # %% import easydiffraction as edi +import numpy as np # %% [markdown] # ## 📦 Define Project @@ -103,6 +104,28 @@ print('max:', experiment.data_range.two_theta_max.value) print('inc:', experiment.data_range.two_theta_inc.value) +# %% [markdown] +# ### Set X-ray Polarization Optics +# +# The polarization coefficient defaults to `0.0`. Setting it to a +# nonzero value includes the Lorentz-polarization optics in the +# calculated X-ray intensities. + +# %% +project.analysis.calculate() +unpolarized_intensity = np.asarray(experiment.data.intensity_calc).copy() + +# %% +experiment.instrument.setup_polarization_coefficient = 0.5 +experiment.instrument.setup_monochromator_twotheta = 26.5650511771 + +# %% +project.analysis.calculate() +polarized_intensity = np.asarray(experiment.data.intensity_calc).copy() + +# %% +print('max intensity change:', np.max(np.abs(polarized_intensity - unpolarized_intensity))) + # %% [markdown] # ## 🚀 Perform Calculation # From 0d55940c04f496c931a11d9082038f931ec78dd6 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:20:46 +0200 Subject: [PATCH 09/16] Reach Phase 1 review gate --- docs/dev/plans/xray-cw-polarization-optics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index fceff781a..1ccd5629b 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -214,7 +214,7 @@ Phase 2 (tests + verification): plus regenerated notebook. Commit: `Document X-ray CW polarization optics fields` -- [ ] **P1.7 — Phase 1 review gate.** No-code step. Mark complete and +- [x] **P1.7 — Phase 1 review gate.** No-code step. Mark complete and commit the checklist update alone. Commit: `Reach Phase 1 review gate` @@ -264,7 +264,7 @@ include them only in the `pixi run fix` commit. Leave generated - [x] P1.4 — Bind cryspy natively - [x] P1.5 — Bind crysfml (verify native line first, then fall back) - [x] P1.6 — Demonstrate the new fields in docs -- [ ] P1.7 — Phase 1 review gate +- [x] P1.7 — Phase 1 review gate - [ ] Phase 2 — tests added/updated and all five tasks pass ## Suggested Pull Request From 1211c0e2c05f2c02c7e8a241764ee92108555785 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:26:32 +0200 Subject: [PATCH 10/16] Update instrument support tag example --- src/easydiffraction/analysis/calculators/support.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/easydiffraction/analysis/calculators/support.py b/src/easydiffraction/analysis/calculators/support.py index a24e7742b..65d6f0571 100644 --- a/src/easydiffraction/analysis/calculators/support.py +++ b/src/easydiffraction/analysis/calculators/support.py @@ -33,7 +33,7 @@ class SupportEntry: Attributes ---------- instrument_tag : str - The instrument category tag (for example ``'cwl-pd'``). + The instrument category tag (for example ``'cwl-pd-neutron'``). description : str One-line human-readable description of the instrument. compatibility : Compatibility From ff60b3ddd52791b4ee06763abba9970475cc56bd Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:26:54 +0200 Subject: [PATCH 11/16] Clarify CW powder instrument base initialization --- .../datablocks/experiment/categories/instrument/cwl.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py b/src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py index e3ccbd58e..17485ebaa 100644 --- a/src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py +++ b/src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py @@ -171,7 +171,7 @@ class CwlPdInstrumentBase(CwlInstrumentBase): """Base class for CW powder diffractometers.""" def __init__(self) -> None: - """Initialize the CW powder diffractometer.""" + """Initialize the CW powder diffractometer base.""" super().__init__() self._calib_twotheta_offset: Parameter = Parameter( From 015b0b5ef99f546ec25207e5f5fd503b24c34816 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:34:22 +0200 Subject: [PATCH 12/16] Add X-ray CW polarization optics tests --- .../analysis/calculators/test_crysfml.py | 26 +++++++ .../analysis/calculators/test_cryspy.py | 59 +++++++++++++- .../analysis/calculators/test_polarization.py | 59 ++++++++++++++ .../analysis/calculators/test_support.py | 7 +- .../categories/instrument/test_cwl.py | 78 +++++++++++++++++-- .../categories/instrument/test_factory.py | 34 +++++++- .../easydiffraction/io/cif/test_serialize.py | 12 +-- 7 files changed, 253 insertions(+), 22 deletions(-) create mode 100644 tests/unit/easydiffraction/analysis/calculators/test_polarization.py diff --git a/tests/unit/easydiffraction/analysis/calculators/test_crysfml.py b/tests/unit/easydiffraction/analysis/calculators/test_crysfml.py index ce4a4eda3..735f152a4 100644 --- a/tests/unit/easydiffraction/analysis/calculators/test_crysfml.py +++ b/tests/unit/easydiffraction/analysis/calculators/test_crysfml.py @@ -49,6 +49,32 @@ def test_crysfml_calculate_pattern_applies_absorption(monkeypatch): assert not np.allclose(out, raw) +def test_crysfml_calculate_pattern_applies_polarization(monkeypatch): + from easydiffraction.analysis.calculators import polarization + from easydiffraction.analysis.calculators.crysfml import CrysfmlCalculator + from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdXrayInstrument + + calc = CrysfmlCalculator() + x = np.array([0.0, 45.0, 90.0]) + instrument = CwlPdXrayInstrument() + instrument.setup_polarization_coefficient = 0.5 + instrument.setup_monochromator_twotheta = 60.0 + experiment = SimpleNamespace( + name='exp', + instrument=instrument, + data=SimpleNamespace(x=x), + ) + raw = [100.0, 100.0, 100.0] + monkeypatch.setattr(calc, '_crysfml_dict', lambda s, e: {}) + monkeypatch.setattr(calc, '_calculate_adjusted_pattern', lambda d, e: list(raw)) + + out = calc.calculate_pattern(None, experiment) + + expected = polarization.apply(raw, experiment) + assert np.allclose(out, expected) + assert not np.allclose(out, raw) + + def test_crysfml_calculate_pattern_preserves_empty_no_data(monkeypatch): from easydiffraction.analysis.calculators.crysfml import CrysfmlCalculator diff --git a/tests/unit/easydiffraction/analysis/calculators/test_cryspy.py b/tests/unit/easydiffraction/analysis/calculators/test_cryspy.py index 0ae584840..e9aae5e57 100644 --- a/tests/unit/easydiffraction/analysis/calculators/test_cryspy.py +++ b/tests/unit/easydiffraction/analysis/calculators/test_cryspy.py @@ -92,7 +92,7 @@ def test_tof_pseudo_voigt_cif_section_uses_non_convoluted_peak_shape(): def test_cwl_cif_instrument_section_emits_sycos_sysin(): import easydiffraction.analysis.calculators.cryspy as MUT - from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdInstrument + from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument from easydiffraction.datablocks.experiment.item.enums import BeamModeEnum from easydiffraction.datablocks.experiment.item.enums import SampleFormEnum @@ -100,7 +100,7 @@ def test_cwl_cif_instrument_section_emits_sycos_sysin(): beam_mode=SimpleNamespace(value=BeamModeEnum.CONSTANT_WAVELENGTH), sample_form=SimpleNamespace(value=SampleFormEnum.POWDER), ) - instrument = CwlPdInstrument() + instrument = CwlPdNeutronInstrument() instrument.calib_sample_displacement = 0.05 instrument.calib_sample_transparency = 0.09 @@ -112,6 +112,29 @@ def test_cwl_cif_instrument_section_emits_sycos_sysin(): assert '_setup_offset_SySin 0.09' in cif_text +def test_cwl_cif_instrument_section_emits_xray_polarization_setup(): + import easydiffraction.analysis.calculators.cryspy as MUT + from easydiffraction.analysis.calculators import polarization + from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdXrayInstrument + from easydiffraction.datablocks.experiment.item.enums import BeamModeEnum + from easydiffraction.datablocks.experiment.item.enums import SampleFormEnum + + expt_type = SimpleNamespace( + beam_mode=SimpleNamespace(value=BeamModeEnum.CONSTANT_WAVELENGTH), + sample_form=SimpleNamespace(value=SampleFormEnum.POWDER), + ) + instrument = CwlPdXrayInstrument() + instrument.setup_polarization_coefficient = 0.5 + instrument.setup_monochromator_twotheta = 60.0 + + cif_lines: list[str] = [] + MUT._cif_instrument_section(cif_lines, expt_type, instrument) + cif_text = '\n'.join(cif_lines) + + assert '_setup_K 0.5' in cif_text + assert f'_setup_cthm {polarization.monochromator_cthm(60.0)}' in cif_text + + def test_update_experiment_in_cryspy_dict_sets_sycos_sysin(): from easydiffraction.analysis.calculators.cryspy import CryspyCalculator @@ -152,6 +175,38 @@ def test_update_experiment_in_cryspy_dict_tolerates_missing_sycos_keys(): assert 'offset_sycos' not in cryspy_dict['pd_exp'] +def test_update_experiment_in_cryspy_dict_sets_polarization_keys(): + from easydiffraction.analysis.calculators import polarization + from easydiffraction.analysis.calculators.cryspy import CryspyCalculator + from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdXrayInstrument + from easydiffraction.datablocks.experiment.item.enums import BeamModeEnum + from easydiffraction.datablocks.experiment.item.enums import SampleFormEnum + + instrument = CwlPdXrayInstrument() + instrument.setup_polarization_coefficient = 0.5 + instrument.setup_monochromator_twotheta = 60.0 + experiment = _cwl_experiment_stub() + experiment.instrument = instrument + experiment.experiment_type = SimpleNamespace( + sample_form=SimpleNamespace(value=SampleFormEnum.POWDER), + beam_mode=SimpleNamespace(value=BeamModeEnum.CONSTANT_WAVELENGTH), + ) + cryspy_dict = { + 'pd_exp': { + 'offset_ttheta': [0.0], + 'wavelength': [0.0], + 'k': [0.0], + 'cthm': [0.0], + 'resolution_parameters': [0.0] * 5, + } + } + + CryspyCalculator._update_experiment_in_cryspy_dict(cryspy_dict, experiment) + + assert cryspy_dict['pd_exp']['k'][0] == 0.5 + assert cryspy_dict['pd_exp']['cthm'][0] == polarization.monochromator_cthm(60.0) + + def test_update_structure_zeroes_biso_for_anisotropic_atoms(): from easydiffraction.analysis.calculators.cryspy import CryspyCalculator from easydiffraction.datablocks.structure.item.base import Structure diff --git a/tests/unit/easydiffraction/analysis/calculators/test_polarization.py b/tests/unit/easydiffraction/analysis/calculators/test_polarization.py new file mode 100644 index 000000000..29d525f04 --- /dev/null +++ b/tests/unit/easydiffraction/analysis/calculators/test_polarization.py @@ -0,0 +1,59 @@ +# SPDX-FileCopyrightText: 2026 EasyScience contributors +# SPDX-License-Identifier: BSD-3-Clause +"""Unit tests for the Lorentz-polarization helper.""" + +from __future__ import annotations + +from types import SimpleNamespace + +import numpy as np +import pytest + +from easydiffraction.analysis.calculators import polarization +from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument +from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdXrayInstrument + + +def test_monochromator_cthm_zero_angle_is_one(): + assert polarization.monochromator_cthm(0.0) == pytest.approx(1.0) + + +def test_monochromator_cthm_matches_cos_squared_identity(): + assert polarization.monochromator_cthm(60.0) == pytest.approx(0.25) + + +def test_lp_factor_is_one_when_coefficient_is_zero(): + two_theta = np.array([0.0, 45.0, 90.0]) + + factor = polarization.lp_factor(two_theta, 0.0, 0.25) + + np.testing.assert_allclose(factor, np.ones_like(two_theta)) + + +def test_apply_is_noop_for_neutron_instrument(): + y = np.array([1.0, 2.0, 3.0]) + experiment = SimpleNamespace( + instrument=CwlPdNeutronInstrument(), + data=SimpleNamespace(x=np.array([10.0, 20.0, 30.0])), + ) + + result = polarization.apply(y, experiment) + + assert result is y + + +def test_apply_multiplies_xray_pattern(): + y = np.array([2.0, 2.0, 2.0]) + instrument = CwlPdXrayInstrument() + instrument.setup_polarization_coefficient = 0.5 + instrument.setup_monochromator_twotheta = 60.0 + two_theta = np.array([0.0, 45.0, 90.0]) + experiment = SimpleNamespace( + instrument=instrument, + data=SimpleNamespace(x=two_theta), + ) + + result = polarization.apply(y, experiment) + + expected = y * polarization.lp_factor(two_theta, 0.5, 0.25) + np.testing.assert_allclose(result, expected) diff --git a/tests/unit/easydiffraction/analysis/calculators/test_support.py b/tests/unit/easydiffraction/analysis/calculators/test_support.py index 0f335ed7c..196172a29 100644 --- a/tests/unit/easydiffraction/analysis/calculators/test_support.py +++ b/tests/unit/easydiffraction/analysis/calculators/test_support.py @@ -27,14 +27,15 @@ def test_matrix_entries_are_well_typed(): assert all(isinstance(c, CalculatorEnum) for c in entry.calculators) -def test_cwl_pd_supports_all_three_engines(): +def test_cwl_pd_bragg_instruments_support_bragg_engines_only(): from easydiffraction.analysis.calculators.support import calculator_support_matrix from easydiffraction.datablocks.experiment.item.enums import CalculatorEnum by_tag = {e.instrument_tag: e for e in calculator_support_matrix()} - expected = {CalculatorEnum.CRYSPY, CalculatorEnum.CRYSFML, CalculatorEnum.PDFFIT} - assert expected <= by_tag['cwl-pd'].calculators + expected = frozenset({CalculatorEnum.CRYSPY, CalculatorEnum.CRYSFML}) + assert by_tag['cwl-pd-neutron'].calculators == expected + assert by_tag['cwl-pd-xray'].calculators == expected def test_cwl_sc_supports_cryspy_only(): diff --git a/tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_cwl.py b/tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_cwl.py index c044c53ab..c0562a318 100644 --- a/tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_cwl.py +++ b/tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_cwl.py @@ -1,15 +1,18 @@ # SPDX-FileCopyrightText: 2026 EasyScience contributors # SPDX-License-Identifier: BSD-3-Clause +from __future__ import annotations + import pytest from easydiffraction.core.variable import NumericDescriptor -from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdInstrument +from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument +from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdXrayInstrument from easydiffraction.utils.logging import Logger def test_cwl_instrument_parameters_settable(): - instr = CwlPdInstrument() + instr = CwlPdNeutronInstrument() instr.setup_wavelength = 2.0 instr.calib_twotheta_offset = 0.1 instr.calib_sample_displacement = 0.05 @@ -21,7 +24,7 @@ def test_cwl_instrument_parameters_settable(): def test_cwl_sample_corrections_default_to_zero_in_degrees(): - instr = CwlPdInstrument() + instr = CwlPdNeutronInstrument() # SyCos/SySin corrections are off by default (no peak-position shift). assert instr.calib_sample_displacement.value == 0.0 assert instr.calib_sample_transparency.value == 0.0 @@ -31,7 +34,7 @@ def test_cwl_sample_corrections_default_to_zero_in_degrees(): def test_cwl_second_wavelength_defaults_off(): - instr = CwlPdInstrument() + instr = CwlPdNeutronInstrument() # No second component by default: monochromatic, as before. assert instr.setup_wavelength_2.value == 0.0 assert instr.setup_wavelength_2_to_1_ratio.value == 0.0 @@ -39,7 +42,7 @@ def test_cwl_second_wavelength_defaults_off(): def test_cwl_second_wavelength_settable(): - instr = CwlPdInstrument() + instr = CwlPdNeutronInstrument() instr.setup_wavelength_2 = 1.5444 instr.setup_wavelength_2_to_1_ratio = 0.5 assert instr.setup_wavelength_2.value == 1.5444 @@ -47,7 +50,7 @@ def test_cwl_second_wavelength_settable(): def test_cwl_second_wavelength_fields_are_non_refinable(): - instr = CwlPdInstrument() + instr = CwlPdNeutronInstrument() # Placeholder fields are NumericDescriptors, not refinable # Parameters, so a fit cannot silently move a value no engine # consumes yet (NumericDescriptor has no `free` flag). @@ -61,7 +64,7 @@ def test_cwl_second_wavelength_fields_are_non_refinable(): def test_cwl_second_wavelength_ratio_rejects_out_of_range(monkeypatch): monkeypatch.setattr(Logger, '_reaction', Logger.Reaction.RAISE, raising=True) - instr = CwlPdInstrument() + instr = CwlPdNeutronInstrument() # Ratio is a relative intensity bounded to [0, 1]. with pytest.raises(TypeError): instr.setup_wavelength_2_to_1_ratio = 1.5 @@ -70,3 +73,64 @@ def test_cwl_second_wavelength_ratio_rejects_out_of_range(monkeypatch): # A negative second wavelength is rejected too. with pytest.raises(TypeError): instr.setup_wavelength_2 = -1.0 + + +def test_cwl_xray_polarization_optics_defaults_off(): + instr = CwlPdXrayInstrument() + + assert isinstance(instr.setup_polarization_coefficient, NumericDescriptor) + assert isinstance(instr.setup_monochromator_twotheta, NumericDescriptor) + assert instr.setup_polarization_coefficient.value == 0.0 + assert instr.setup_monochromator_twotheta.value == 0.0 + assert not hasattr(instr.setup_polarization_coefficient, 'free') + assert not hasattr(instr.setup_monochromator_twotheta, 'free') + + +def test_cwl_xray_polarization_optics_settable(): + instr = CwlPdXrayInstrument() + + instr.setup_polarization_coefficient = 0.5 + instr.setup_monochromator_twotheta = 26.565 + + assert instr.setup_polarization_coefficient.value == 0.5 + assert instr.setup_monochromator_twotheta.value == 26.565 + + +def test_cwl_xray_polarization_optics_reject_invalid_values(monkeypatch): + monkeypatch.setattr(Logger, '_reaction', Logger.Reaction.RAISE, raising=True) + instr = CwlPdXrayInstrument() + + with pytest.raises(TypeError): + instr.setup_polarization_coefficient = -0.1 + with pytest.raises(TypeError): + instr.setup_polarization_coefficient = 1.1 + with pytest.raises(TypeError): + instr.setup_polarization_coefficient = 'bad' + with pytest.raises(TypeError): + instr.setup_monochromator_twotheta = -0.1 + with pytest.raises(TypeError): + instr.setup_monochromator_twotheta = 180.0 + with pytest.raises(TypeError): + instr.setup_monochromator_twotheta = 'bad' + + +def test_cwl_neutron_instrument_has_no_polarization_optics(): + instr = CwlPdNeutronInstrument() + + assert not hasattr(instr, 'setup_polarization_coefficient') + assert not hasattr(instr, 'setup_monochromator_twotheta') + + +def test_cwl_xray_polarization_optics_round_trip_through_cif(): + import gemmi + + instr = CwlPdXrayInstrument() + instr.setup_polarization_coefficient = 0.5 + instr.setup_monochromator_twotheta = 26.565 + + block = gemmi.cif.read_string('data_x\n' + instr.as_cif + '\n').sole_block() + restored = CwlPdXrayInstrument() + restored.from_cif(block) + + assert restored.setup_polarization_coefficient.value == 0.5 + assert restored.setup_monochromator_twotheta.value == 26.565 diff --git a/tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_factory.py b/tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_factory.py index 7a8b40a30..8f8ac4eab 100644 --- a/tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_factory.py +++ b/tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_factory.py @@ -1,6 +1,8 @@ # SPDX-FileCopyrightText: 2026 EasyScience contributors # SPDX-License-Identifier: BSD-3-Clause +from __future__ import annotations + import pytest @@ -10,18 +12,20 @@ def test_instrument_factory_default_and_errors(): InstrumentFactory, ) from easydiffraction.datablocks.experiment.item.enums import BeamModeEnum + from easydiffraction.datablocks.experiment.item.enums import RadiationProbeEnum from easydiffraction.datablocks.experiment.item.enums import SampleFormEnum + from easydiffraction.datablocks.experiment.item.enums import ScatteringTypeEnum except ImportError as e: # pragma: no cover - environment-specific circular import pytest.skip(f'InstrumentFactory import triggers circular import in this context: {e}') return # By tag - inst = InstrumentFactory.create('cwl-pd') - assert inst.__class__.__name__ == 'CwlPdInstrument' + inst = InstrumentFactory.create('cwl-pd-neutron') + assert inst.__class__.__name__ == 'CwlPdNeutronInstrument' # By tag - inst2 = InstrumentFactory.create('cwl-pd') - assert inst2.__class__.__name__ == 'CwlPdInstrument' + inst2 = InstrumentFactory.create('cwl-pd-xray') + assert inst2.__class__.__name__ == 'CwlPdXrayInstrument' inst3 = InstrumentFactory.create('tof-pd') assert inst3.__class__.__name__ == 'TofPdInstrument' @@ -32,9 +36,31 @@ def test_instrument_factory_default_and_errors(): ) assert tag == 'tof-pd' + neutron_tag = InstrumentFactory.default_tag( + beam_mode=BeamModeEnum.CONSTANT_WAVELENGTH, + sample_form=SampleFormEnum.POWDER, + scattering_type=ScatteringTypeEnum.BRAGG, + radiation_probe=RadiationProbeEnum.NEUTRON, + ) + assert neutron_tag == 'cwl-pd-neutron' + + xray_tag = InstrumentFactory.default_tag( + beam_mode=BeamModeEnum.CONSTANT_WAVELENGTH, + sample_form=SampleFormEnum.POWDER, + scattering_type=ScatteringTypeEnum.BRAGG, + radiation_probe=RadiationProbeEnum.XRAY, + ) + assert xray_tag == 'cwl-pd-xray' + # Invalid tag with pytest.raises( ValueError, match=r"Unsupported type: 'nonexistent'\. Supported: .*", ): InstrumentFactory.create('nonexistent') + + with pytest.raises( + ValueError, + match=r"Unsupported type: 'cwl-pd'\. Supported: .*", + ): + InstrumentFactory.create('cwl-pd') diff --git a/tests/unit/easydiffraction/io/cif/test_serialize.py b/tests/unit/easydiffraction/io/cif/test_serialize.py index eca59af05..8aba9c215 100644 --- a/tests/unit/easydiffraction/io/cif/test_serialize.py +++ b/tests/unit/easydiffraction/io/cif/test_serialize.py @@ -318,15 +318,15 @@ def test_beta_atom_round_trips_through_cif(): def test_cwl_second_wavelength_round_trips_through_cif(): import gemmi - from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdInstrument + from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument - instr = CwlPdInstrument() + instr = CwlPdNeutronInstrument() instr.setup_wavelength = 1.5406 instr.setup_wavelength_2 = 1.5444 instr.setup_wavelength_2_to_1_ratio = 0.5 block = gemmi.cif.read_string('data_x\n' + instr.as_cif + '\n').sole_block() - restored = CwlPdInstrument() + restored = CwlPdNeutronInstrument() restored.from_cif(block) assert restored.setup_wavelength_2.value == 1.5444 @@ -336,15 +336,15 @@ def test_cwl_second_wavelength_round_trips_through_cif(): def test_cwl_disabled_second_wavelength_preserves_value_through_cif(): import gemmi - from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdInstrument + from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument # Disabled state (ratio == 0) still persists the recorded λ₂. - instr = CwlPdInstrument() + instr = CwlPdNeutronInstrument() instr.setup_wavelength = 1.5406 instr.setup_wavelength_2 = 1.5444 block = gemmi.cif.read_string('data_x\n' + instr.as_cif + '\n').sole_block() - restored = CwlPdInstrument() + restored = CwlPdNeutronInstrument() restored.from_cif(block) assert restored.setup_wavelength_2.value == 1.5444 From 42eb9971bc8d0bc0a18f79e9798d7c3bd832e214 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 00:37:05 +0200 Subject: [PATCH 13/16] Apply Phase 2 formatting updates --- .../accepted/xray-cw-polarization-optics.md | 137 +++++------ docs/dev/adrs/index.md | 2 +- docs/dev/package-structure/full.md | 5 +- docs/dev/package-structure/short.md | 1 + docs/dev/plans/xray-cw-polarization-optics.md | 219 +++++++++--------- docs/docs/tutorials/simulate-nacl-xray.ipynb | 5 +- docs/docs/tutorials/simulate-nacl-xray.py | 3 +- .../analysis/calculators/cryspy.py | 7 +- .../analysis/calculators/polarization.py | 16 +- .../analysis/calculators/test_cryspy.py | 4 +- .../easydiffraction/io/cif/test_serialize.py | 8 +- 11 files changed, 213 insertions(+), 194 deletions(-) diff --git a/docs/dev/adrs/accepted/xray-cw-polarization-optics.md b/docs/dev/adrs/accepted/xray-cw-polarization-optics.md index 5796c4b59..d406b69d8 100644 --- a/docs/dev/adrs/accepted/xray-cw-polarization-optics.md +++ b/docs/dev/adrs/accepted/xray-cw-polarization-optics.md @@ -19,12 +19,12 @@ This ADR follows the conventions in Constant-wavelength X-ray powder calculations need the same Lorentz-polarization controls that FullProf exposes on the PCR -`Lambda1 Lambda2 Ratio Bkpos Wdt Cthm muR AsyLim Rpolarz 2nd-muR` -line. The two relevant values are: +`Lambda1 Lambda2 Ratio Bkpos Wdt Cthm muR AsyLim Rpolarz 2nd-muR` line. +The two relevant values are: -| FullProf field | Backend field | Meaning | -| -------------- | ------------- | ------- | -| `Rpolarz` | Cryspy `_setup_K` | Polarization coefficient in the CW Lorentz-polarization factor. | +| FullProf field | Backend field | Meaning | +| -------------- | -------------------- | ----------------------------------------------------------------- | +| `Rpolarz` | Cryspy `_setup_K` | Polarization coefficient in the CW Lorentz-polarization factor. | | `Cthm` | Cryspy `_setup_cthm` | `cos²(2θm)`, where `2θm` is the pre-specimen monochromator angle. | These names are calculator-oriented and should not become the public @@ -53,17 +53,17 @@ candidates, consistent with ### Backend support already present The Cryspy `Setup` item already declares both fields as optional, -non-refinable descriptors (`C_item_loop_classes/cl_1_setup.py`): -`k` (CIF `_setup_K`) and `cthm` (CIF `_setup_cthm`), with defaults -`k = 0.0` and `cthm = 0.91`. Neither is in Cryspy's refinable-attribute -set, so binding them is a value-injection task, not a backend -extension. (Note the FullProf diagnostic below uses `Cthm = 0.8`, which -differs from Cryspy's `0.91` default — defaults must be set explicitly, -not inherited from the backend.) CrysFML's current Python wrapper has no -equivalent declared field (see Decision 5). - -The existing CW instrument category is a single class, -`CwlPdInstrument` (factory tag `cwl-pd`, +non-refinable descriptors (`C_item_loop_classes/cl_1_setup.py`): `k` +(CIF `_setup_K`) and `cthm` (CIF `_setup_cthm`), with defaults `k = 0.0` +and `cthm = 0.91`. Neither is in Cryspy's refinable-attribute set, so +binding them is a value-injection task, not a backend extension. (Note +the FullProf diagnostic below uses `Cthm = 0.8`, which differs from +Cryspy's `0.91` default — defaults must be set explicitly, not inherited +from the backend.) CrysFML's current Python wrapper has no equivalent +declared field (see Decision 5). + +The existing CW instrument category is a single class, `CwlPdInstrument` +(factory tag `cwl-pd`, `src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py`), serving **both** neutron and X-ray, **both** `bragg` and `total` scattering, across the `cryspy`, `crysfml`, and `pdffit` calculators. @@ -74,8 +74,8 @@ the model this ADR follows for declaration style. ### Diagnostic evidence -`docs/docs/verification/pd-xray-pbso4-single-polarized-wdt48.py` -adds a focused diagnostic reference generated from +`docs/docs/verification/pd-xray-pbso4-single-polarized-wdt48.py` adds a +focused diagnostic reference generated from `pbsox_single_polarized_wdt48.pcr`: - `Lambda1 = Lambda2 = 1.540560` @@ -84,15 +84,15 @@ adds a focused diagnostic reference generated from - `Rpolarz = 0.5` - `Cthm = 0.8` -The current EasyDiffraction adapters do not expose or bind these -optics fields. Against that FullProf reference the current results are: +The current EasyDiffraction adapters do not expose or bind these optics +fields. Against that FullProf reference the current results are: -| Engine | State | Profile diff | Max deviation | Area ratio | Correlation | -| ------ | ----- | ------------ | ------------- | ---------- | ----------- | -| cryspy | raw | 33.65 % | 33.54 % | 0.7537 | 0.9931 | -| cryspy | scale + U/V/W/Y refined | 10.20 % | 6.48 % | 1.1222 | 0.9950 | -| crysfml | raw | 36.18 % | 34.63 % | 0.6859 | 0.9895 | -| crysfml | scale + U/V/W/Y refined | 10.08 % | 6.69 % | 1.1156 | 0.9951 | +| Engine | State | Profile diff | Max deviation | Area ratio | Correlation | +| ------- | ----------------------- | ------------ | ------------- | ---------- | ----------- | +| cryspy | raw | 33.65 % | 33.54 % | 0.7537 | 0.9931 | +| cryspy | scale + U/V/W/Y refined | 10.20 % | 6.48 % | 1.1222 | 0.9950 | +| crysfml | raw | 36.18 % | 34.63 % | 0.6859 | 0.9895 | +| crysfml | scale + U/V/W/Y refined | 10.08 % | 6.69 % | 1.1156 | 0.9951 | The large raw area mismatch is expected: the FullProf reference contains an active X-ray polarization correction, while EasyDiffraction currently @@ -117,8 +117,8 @@ for neutron CW powder experiments, where the polarization coefficient is identically zero (unlike sample absorption, which is physically real for both probes — so the [`model-sample-absorption.md`](../accepted/model-sample-absorption.md) -single-shared-category precedent does not transfer cleanly here). Use the -existing `Compatibility.radiation_probe` axis and register separate +single-shared-category precedent does not transfer cleanly here). Use +the existing `Compatibility.radiation_probe` axis and register separate instrument classes: - `CwlPdNeutronInstrument` @@ -131,16 +131,15 @@ does not expose them. discriminator in the codebase.** It therefore carries concrete, non-trivial impact that the implementing plan must cover: -- `InstrumentFactory` default rules - (`.../instrument/factory.py`) currently key only on - `(beam_mode, sample_form)`. They must gain `radiation_probe`, and the - single `cwl-pd` tag splits into `cwl-pd-neutron` / `cwl-pd-xray` - (factory tags follow +- `InstrumentFactory` default rules (`.../instrument/factory.py`) + currently key only on `(beam_mode, sample_form)`. They must gain + `radiation_probe`, and the single `cwl-pd` tag splits into + `cwl-pd-neutron` / `cwl-pd-xray` (factory tags follow [`factory-tag-naming.md`](../accepted/factory-tag-naming.md)). - The call site `BraggPdExperiment` (`.../experiment/item/bragg_pd.py`) builds the default instrument tag via - `InstrumentFactory.default_tag(scattering_type, beam_mode, - sample_form)` and must also pass `radiation_probe`. + `InstrumentFactory.default_tag(scattering_type, beam_mode, sample_form)` + and must also pass `radiation_probe`. - Renaming the persisted **instrument** tag is a breaking change to saved projects and to any test/tutorial CIF pinning the instrument `cwl-pd`. The project is in beta (no legacy shims), so the rename is @@ -167,13 +166,13 @@ PDF path does not route through `cwl-pd` and is untouched by the split. unused aspirational metadata, not a live routing path.) The routing matrix the plan must implement: -| scattering | beam_mode | sample_form | radiation_probe | default tag | -| ---------- | --------- | ----------- | --------------- | ----------- | -| bragg | constant wavelength | powder | neutron | `cwl-pd-neutron` | -| bragg | constant wavelength | powder | xray | `cwl-pd-xray` | -| bragg | constant wavelength | single crystal | (any) | `cwl-sc` (unchanged) | -| bragg | time of flight | powder | (any) | `tof-pd` (unchanged) | -| total | constant wavelength | powder | (any) | no factory instrument (unchanged) | +| scattering | beam_mode | sample_form | radiation_probe | default tag | +| ---------- | ------------------- | -------------- | --------------- | --------------------------------- | +| bragg | constant wavelength | powder | neutron | `cwl-pd-neutron` | +| bragg | constant wavelength | powder | xray | `cwl-pd-xray` | +| bragg | constant wavelength | single crystal | (any) | `cwl-sc` (unchanged) | +| bragg | time of flight | powder | (any) | `tof-pd` (unchanged) | +| total | constant wavelength | powder | (any) | no factory instrument (unchanged) | Single crystal (`cwl-sc`) and TOF (`tof-pd`/`tof-sc`) stay probe-neutral: this ADR scopes X-ray optics to powder-Bragg CW, where @@ -187,9 +186,10 @@ classification note above). opt-in.** Even on `CwlPdXrayInstrument`, the polarization coefficient defaults to `0.0` (and the monochromator term is then inert, since the Lorentz-polarization factor reduces to the neutron form when `k = 0` — -see Decision 4). Existing X-ray verification numbers therefore do **not** -change until a user explicitly sets the coefficient. Picking a non-zero -characteristic-radiation default is deferred (see Deferred Work). +see Decision 4). Existing X-ray verification numbers therefore do +**not** change until a user explicitly sets the coefficient. Picking a +non-zero characteristic-radiation default is deferred (see Deferred +Work). ### 3. Use physical public names, non-refinable @@ -204,8 +204,8 @@ experiment.instrument.setup_monochromator_twotheta Do not expose public names `setup_k`, `setup_cthm`, `K`, or `Cthm`. -`setup_polarization_coefficient` is a dimensionless fixed descriptor. -It maps directly to FullProf `Rpolarz` and Cryspy `_setup_K`. It is a +`setup_polarization_coefficient` is a dimensionless fixed descriptor. It +maps directly to FullProf `Rpolarz` and Cryspy `_setup_K`. It is a polarization fraction, so the value is constrained to `[0, 1]` and **defaults to `0.0`** (no correction — the pure opt-in default of Decision 2): @@ -261,8 +261,8 @@ Backend adapters convert the public angle to the backend value: cthm = cos(radians(setup_monochromator_twotheta)) ** 2 ``` -because the public value is the monochromator `2θ` angle and the -backend wants `cos²(2θm)`. This mirrors the established public→backend +because the public value is the monochromator `2θ` angle and the backend +wants `cos²(2θm)`. This mirrors the established public→backend conversion pattern (`_march_r_to_cryspy_g1` in `analysis/calculators/cryspy.py`, where the public March coefficient is inverted before it reaches Cryspy). @@ -290,19 +290,20 @@ adapter only needs to inject values for `CwlPdXrayInstrument`: minimizer iterations do not recompute against stale values; - include these fields in the cache-invalidation surface. -Cryspy thus applies the LP factor **internally**, on its own -`two_theta` grid — it does **not** consume the EasyDiffraction `hh` -multiplier at runtime. The only EasyDiffraction code on the Cryspy path -is the angle→`cthm` conversion (the input it is fed). See Decision 5 for +Cryspy thus applies the LP factor **internally**, on its own `two_theta` +grid — it does **not** consume the EasyDiffraction `hh` multiplier at +runtime. The only EasyDiffraction code on the Cryspy path is the +angle→`cthm` conversion (the input it is fed). See Decision 5 for exactly what is shared and what is not. ### 5. Bind CrysFML through the CFL path or a shared adapter correction -The current CrysFML Python wrapper exposes `patterns_simulation(strings)` -around a CFL parser, and the adapter maps instrument scalars through -`_INSTRUMENT_ATTRIBUTE_MAP` in `analysis/calculators/crysfml.py`. The -bundled CFL examples document `LAMBDA`, `UVWXY`, `ASYM`, `WDT`, and -`Zero_Sy`, but they do not show `Cthm` or `Rpolarz` condition lines. +The current CrysFML Python wrapper exposes +`patterns_simulation(strings)` around a CFL parser, and the adapter maps +instrument scalars through `_INSTRUMENT_ATTRIBUTE_MAP` in +`analysis/calculators/crysfml.py`. The bundled CFL examples document +`LAMBDA`, `UVWXY`, `ASYM`, `WDT`, and `Zero_Sy`, but they do not show +`Cthm` or `Rpolarz` condition lines. Implementation must first verify whether the active CrysFML CFL grammar accepts a native polarization/monochromator line. If it does, the @@ -317,8 +318,8 @@ hh = 1 - k + k * cthm * cos(two_theta) ** 2 y_corrected = hh * y_crysfml ``` -This fallback is acceptable because it is a smooth -Lorentz-polarization envelope on the calculated CW powder pattern. +This fallback is acceptable because it is a smooth Lorentz-polarization +envelope on the calculated CW powder pattern. **What is shared vs. what is not.** Cryspy binds natively (Decision 4) and CrysFML uses the fallback multiplier, so the two engines do **not** @@ -367,8 +368,8 @@ deferral is tracked rather than lost. - `src/easydiffraction/analysis/calculators/cryspy.py` — emit `_setup_K` / `_setup_cthm` (via the shared angle→`cthm` helper), patch cached dicts; native LP, no runtime envelope. -- `src/easydiffraction/analysis/calculators/crysfml.py` — CFL line or the - shared `lp_factor` post-pattern multiplier. +- `src/easydiffraction/analysis/calculators/crysfml.py` — CFL line or + the shared `lp_factor` post-pattern multiplier. - A new shared module (alongside the existing calculator helpers) holding the two pieces from Decision 5: `monochromator_cthm` (angle→ `cthm`, used by both adapters) and `lp_factor` (the `hh` envelope; the @@ -411,8 +412,8 @@ deferral is tracked rather than lost. ### Expose `setup_k` and `setup_cthm` -Rejected. These names are backend/PCR implementation details and are -not discoverable for non-programmer users. +Rejected. These names are backend/PCR implementation details and are not +discoverable for non-programmer users. ### Keep one `CwlPdInstrument` with neutron-neutral defaults @@ -420,7 +421,7 @@ Seriously considered — it is the smaller change and mirrors the single-shared-category mechanism of [`model-sample-absorption.md`](../accepted/model-sample-absorption.md) (fields present everywhere, inert by default). Rejected as the primary -decision because the polarization coefficient is physically *zero* for +decision because the polarization coefficient is physically _zero_ for neutrons, not merely defaulted-off (sample absorption, by contrast, is real for neutrons), so showing the knob on neutron experiments invites meaningless tuning. Recorded as an Open Question because it avoids the @@ -432,9 +433,9 @@ fallback. Rejected for now. There are only two fields and one concrete use case; `experiment.instrument` is the natural category, and introducing an -optics abstraction before a second use case violates the -"don't introduce abstractions before a concrete second use case" -guidance in [`AGENTS.md`](../../../../AGENTS.md). +optics abstraction before a second use case violates the "don't +introduce abstractions before a concrete second use case" guidance in +[`AGENTS.md`](../../../../AGENTS.md). ## Deferred Work diff --git a/docs/dev/adrs/index.md b/docs/dev/adrs/index.md index e74e75f50..4d3804812 100644 --- a/docs/dev/adrs/index.md +++ b/docs/dev/adrs/index.md @@ -42,7 +42,7 @@ folders. | Experiment model | Accepted | Calculation Without Measured Data | Adds a writable `data_range` category so a structure-only experiment is calculable and plottable without loaded data. | [`calculation-without-measured-data.md`](accepted/calculation-without-measured-data.md) | | Experiment model | Accepted | Preferred-Orientation Category | Adds a per-phase March–Dollase preferred-orientation category for textured powder refinement on the CrysPy backend. | [`preferred-orientation-category.md`](accepted/preferred-orientation-category.md) | | Experiment model | Accepted | Model Sample Absorption (Debye–Scherrer, μR) | Switchable `absorption` category applying a calculator-independent cylindrical Hewat A(θ) envelope for powder samples. | [`model-sample-absorption.md`](accepted/model-sample-absorption.md) | -| Experiment model | Accepted | X-ray CW Polarization Optics | Adds discoverable X-ray CW powder instrument fields for FullProf/Cryspy Lorentz-polarization and monochromator optics. | [`xray-cw-polarization-optics.md`](accepted/xray-cw-polarization-optics.md) | +| Experiment model | Accepted | X-ray CW Polarization Optics | Adds discoverable X-ray CW powder instrument fields for FullProf/Cryspy Lorentz-polarization and monochromator optics. | [`xray-cw-polarization-optics.md`](accepted/xray-cw-polarization-optics.md) | | Factories | Accepted | Factory Contracts and Metadata | Standardizes factory construction, metadata, compatibility, and registration behavior. | [`factory-contracts.md`](accepted/factory-contracts.md) | | Naming | Accepted | Factory Tag Naming | Defines canonical factory tag style and standard abbreviations. | [`factory-tag-naming.md`](accepted/factory-tag-naming.md) | | Naming | Accepted | Downloadable Resource Naming | Replaces integer dataset/tutorial ids with stable descriptive slugs and moves presentation order into separate metadata. | [`resource-naming.md`](accepted/resource-naming.md) | diff --git a/docs/dev/package-structure/full.md b/docs/dev/package-structure/full.md index 61a2cbfe1..baa1c2846 100644 --- a/docs/dev/package-structure/full.md +++ b/docs/dev/package-structure/full.md @@ -17,6 +17,7 @@ │ │ │ └── 🏷️ class CalculatorFactory │ │ ├── 📄 pdffit.py │ │ │ └── 🏷️ class PdffitCalculator +│ │ ├── 📄 polarization.py │ │ └── 📄 support.py │ │ └── 🏷️ class SupportEntry │ ├── 📁 categories @@ -373,7 +374,9 @@ │ │ │ │ ├── 📄 cwl.py │ │ │ │ │ ├── 🏷️ class CwlInstrumentBase │ │ │ │ │ ├── 🏷️ class CwlScInstrument -│ │ │ │ │ └── 🏷️ class CwlPdInstrument +│ │ │ │ │ ├── 🏷️ class CwlPdInstrumentBase +│ │ │ │ │ ├── 🏷️ class CwlPdNeutronInstrument +│ │ │ │ │ └── 🏷️ class CwlPdXrayInstrument │ │ │ │ ├── 📄 factory.py │ │ │ │ │ └── 🏷️ class InstrumentFactory │ │ │ │ └── 📄 tof.py diff --git a/docs/dev/package-structure/short.md b/docs/dev/package-structure/short.md index e038ef6e7..5cf80f3a4 100644 --- a/docs/dev/package-structure/short.md +++ b/docs/dev/package-structure/short.md @@ -11,6 +11,7 @@ │ │ ├── 📄 cryspy.py │ │ ├── 📄 factory.py │ │ ├── 📄 pdffit.py +│ │ ├── 📄 polarization.py │ │ └── 📄 support.py │ ├── 📁 categories │ │ ├── 📁 aliases diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index 1ccd5629b..8f2cbe91a 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -32,7 +32,8 @@ Related accepted ADRs consulted: ## Branch and PR - Flat-slug implementation branch off `develop`: - `xray-cw-polarization-optics` (created/checked out by `/draft-impl-1`). + `xray-cw-polarization-optics` (created/checked out by + `/draft-impl-1`). - PR targets `develop`, not `master`. Do not push unless asked. ## Dependencies @@ -49,45 +50,46 @@ injection, not a backend extension. No `pyproject.toml` / `pixi.toml` / 2. The CW **powder Bragg** instrument splits by `radiation_probe`: `CwlPdInstrument` (tag `cwl-pd`) becomes `CwlPdNeutronInstrument` (`cwl-pd-neutron`) and `CwlPdXrayInstrument` (`cwl-pd-xray`). The - X-ray class owns the optics fields; the neutron class does not. Single - crystal (`cwl-sc`), TOF (`tof-pd`/`tof-sc`), and total-scattering PDF - are unchanged. The identically named **data-range** `cwl-pd` tag is - out of scope and stays. Both new classes carry the full **Bragg-only** - compatibility tuple — `sample_form={POWDER}`, - `scattering_type={BRAGG}`, `beam_mode={CONSTANT_WAVELENGTH}`, plus - their `radiation_probe` — and `calculator_support={CRYSPY, CRYSFML}`. - The old class's `TOTAL` / `PDFFIT` metadata is dropped because - total-scattering PDF never routes through this instrument (the ADR - calls those entries unused aspirational metadata); empty - `Compatibility` axes mean "any", so the tuple must be stated in full - to avoid silently matching every sample form / beam mode. + X-ray class owns the optics fields; the neutron class does not. + Single crystal (`cwl-sc`), TOF (`tof-pd`/`tof-sc`), and + total-scattering PDF are unchanged. The identically named + **data-range** `cwl-pd` tag is out of scope and stays. Both new + classes carry the full **Bragg-only** compatibility tuple — + `sample_form={POWDER}`, `scattering_type={BRAGG}`, + `beam_mode={CONSTANT_WAVELENGTH}`, plus their `radiation_probe` — and + `calculator_support={CRYSPY, CRYSFML}`. The old class's `TOTAL` / + `PDFFIT` metadata is dropped because total-scattering PDF never + routes through this instrument (the ADR calls those entries unused + aspirational metadata); empty `Compatibility` axes mean "any", so the + tuple must be stated in full to avoid silently matching every sample + form / beam mode. 3. Public names are physical and non-refinable (`NumericDescriptor`): - `setup_polarization_coefficient` — dimensionless, default `0.0`, range `[0, 1]` → cryspy `_setup_K` / FullProf `Rpolarz`. - `setup_monochromator_twotheta` — degrees, default `0.0`, range - `[0, 180)` → backend `cthm = cos²(radians(2θm))` (default `0.0°` - ⇒ `cthm = 1.0`, "no monochromator"). + `[0, 180)` → backend `cthm = cos²(radians(2θm))` (default `0.0°` ⇒ + `cthm = 1.0`, "no monochromator"). 4. **cryspy binds natively**: emit `_setup_K` / `_setup_cthm` in the generated CIF (and patch the cached dict like other instrument scalars). cryspy applies the Lorentz-polarization factor internally; the EasyDiffraction multiplier is **not** applied to cryspy output. -5. **crysfml binds via a post-pattern multiplier**, but only *after* - the ADR Decision 5 gate is satisfied: the implementer first verifies +5. **crysfml binds via a post-pattern multiplier**, but only _after_ the + ADR Decision 5 gate is satisfied: the implementer first verifies (static source/CFL-grammar inspection) whether a native polarization/monochromator CFL line exists. If one does, that native line is emitted (and the ADR/plan updated); otherwise — the expected - outcome from the bundled CFL examples — `y *= lp_factor(two_theta, k, - cthm)` is applied after the engine returns, in a shared, unit-tested - helper. Two shared pieces: + outcome from the bundled CFL examples — + `y *= lp_factor(two_theta, k, cthm)` is applied after the engine + returns, in a shared, unit-tested helper. Two shared pieces: `monochromator_cthm(angle)` (used by both adapters to produce the backend `cthm`) and `lp_factor(two_theta, k, cthm)` (crysfml runtime - multiplier + the oracle that verifies cryspy's native output). - P1.5 inspection found lower-level CrysFML Lorentz routines with - optional `cthm`/`rkk` arguments, but the Python - `cw_powder_pattern_from_dict` wrapper reads no dictionary keys for - them, so the planned fallback multiplier is used. -6. Defaults are neutron-neutral: with the coefficient at `0.0`, - `hh = 1` and results are unchanged until a user opts in. + multiplier + the oracle that verifies cryspy's native output). P1.5 + inspection found lower-level CrysFML Lorentz routines with optional + `cthm`/`rkk` arguments, but the Python `cw_powder_pattern_from_dict` + wrapper reads no dictionary keys for them, so the planned fallback + multiplier is used. +6. Defaults are neutron-neutral: with the coefficient at `0.0`, `hh = 1` + and results are unchanged until a user opts in. ## Open questions @@ -119,9 +121,9 @@ Phase 1 (code + docs): `InstrumentFactory.default_tag(...)`. - `src/easydiffraction/analysis/calculators/polarization.py` — **new** shared helper: `monochromator_cthm`, `lp_factor`, `apply`. -- `src/easydiffraction/analysis/calculators/cryspy.py` — emit - `_setup_K` / `_setup_cthm` in `_cif_instrument_section`; optional - cached-dict patch in `_update_experiment_in_cryspy_dict`. +- `src/easydiffraction/analysis/calculators/cryspy.py` — emit `_setup_K` + / `_setup_cthm` in `_cif_instrument_section`; optional cached-dict + patch in `_update_experiment_in_cryspy_dict`. - `src/easydiffraction/analysis/calculators/crysfml.py` — apply `polarization.apply(y, experiment)` on the return path (~line 170). - A tutorial/verification source `.py` (P1.6, pending open question 1) @@ -140,83 +142,86 @@ Phase 2 (tests + verification): ## Implementation steps (Phase 1) - [x] **P1.1 — Promote the ADR to `accepted/`.** The ADR was moved into - `docs/dev/adrs/accepted/`, its status was set to `Accepted`, the - existing **Experiment model** row in `docs/dev/adrs/index.md` was - repointed to `accepted/xray-cw-polarization-optics.md`, this plan's - `## ADR` link was updated, and remaining stale path references were - cleared. - Commit: `Promote xray-cw-polarization-optics ADR to accepted` - -- [x] **P1.2 — Split the CW powder instrument by radiation probe.** - In `cwl.py`, extract a `CwlPdInstrumentBase(CwlInstrumentBase)` holding - the existing `calib_*` fields and properties. Add - `CwlPdNeutronInstrument` (tag `cwl-pd-neutron`) and - `CwlPdXrayInstrument` (tag `cwl-pd-xray`). Give **both** the full - Bragg-only `Compatibility`: - `sample_form=frozenset({SampleFormEnum.POWDER})`, - `scattering_type=frozenset({ScatteringTypeEnum.BRAGG})`, - `beam_mode=frozenset({BeamModeEnum.CONSTANT_WAVELENGTH})`, and - `radiation_probe=frozenset({RadiationProbeEnum.NEUTRON})` / - `{RadiationProbeEnum.XRAY})` respectively; and - `calculator_support=CalculatorSupport(calculators=frozenset( - {CalculatorEnum.CRYSPY, CalculatorEnum.CRYSFML}))` — **no `PDFFIT`, - no `TOTAL`**. On the X-ray class declare the two `NumericDescriptor`s - (with `DisplayHandler`, `AttributeSpec` defaults/validators, and - `TagSpec` exactly as ADR Decision 3) plus their getter/setter - properties (mirror the `setup_wavelength_2` property style). In - `factory.py` replace the `cwl-pd` rule with two rules keyed on - `radiation_probe`. In `bragg_pd.py` pass `radiation_probe=...` to - `default_tag(...)`. Update `instrument/__init__.py` registration. Leave - the data-range `cwl-pd` tag untouched. - Commit: `Split CW powder instrument by radiation probe` - -- [x] **P1.3 — Add the shared Lorentz-polarization helper.** - New module `analysis/calculators/polarization.py` mirroring - `absorption.py`: `monochromator_cthm(twotheta_deg) -> float` - (`cos²(radians(twotheta))`), `lp_factor(two_theta, k, cthm) -> - np.ndarray` (`1 - k + k*cthm*cos²(two_theta)`), and `apply(y, - experiment) -> np.ndarray` that no-ops unless the instrument exposes - `setup_polarization_coefficient` with a non-zero value, otherwise - multiplies `y` by `lp_factor` over the experiment's 2θ grid. - Commit: `Add shared Lorentz-polarization helper` - -- [x] **P1.4 — Bind cryspy natively.** - In `cryspy.py` `_cif_instrument_section`, within the existing powder - branch, when `hasattr(instrument, 'setup_polarization_coefficient')` - append `_setup_K ` and - `_setup_cthm ` - (import `monochromator_cthm` from P1.3). Do **not** apply - `polarization.apply` to cryspy output. If the cached working dict in - `_update_experiment_in_cryspy_dict` exposes the keys, patch them with a - presence guard (mirroring `offset_sycos`); otherwise document reliance - on the CIF-build path + cache rebuild (open question 2). - Commit: `Emit cryspy polarization setup for X-ray CW powder` - -- [x] **P1.5 — Bind crysfml (verify native line first, then fall back).** - Satisfy the ADR Decision 5 gate explicitly: statically inspect the - active CrysFML CFL grammar / bundled examples for a native - polarization or monochromator line. **If one exists**, emit that native - line and update the ADR/plan to record it (or stop and ask if it - reopens scope). **If none exists** (the expected outcome), apply - `polarization.apply(y, experiment)` on the crysfml return path - alongside the existing `absorption_correction.apply(...)` - (~`crysfml.py:170`); no-op for neutron / zero coefficient. State the - inspection result in the commit body. - Commit: `Apply Lorentz-polarization envelope on crysfml CW powder` - -- [x] **P1.6 — Demonstrate the new fields in docs.** - Per open question 1: either add the - `pd-xray-pbso4-single-polarized-wdt48.py` verification source (if the - FullProf reference data is available) or set - `setup_polarization_coefficient` / `setup_monochromator_twotheta` in an - existing X-ray CW source and show the pattern responds. Edit the `.py` - source only, then run `pixi run notebook-prepare` and commit the source - plus regenerated notebook. - Commit: `Document X-ray CW polarization optics fields` + `docs/dev/adrs/accepted/`, its status was set to `Accepted`, the + existing **Experiment model** row in `docs/dev/adrs/index.md` was + repointed to `accepted/xray-cw-polarization-optics.md`, this + plan's `## ADR` link was updated, and remaining stale path + references were cleared. Commit: + `Promote xray-cw-polarization-optics ADR to accepted` + +- [x] **P1.2 — Split the CW powder instrument by radiation probe.** In + `cwl.py`, extract a `CwlPdInstrumentBase(CwlInstrumentBase)` + holding the existing `calib_*` fields and properties. Add + `CwlPdNeutronInstrument` (tag `cwl-pd-neutron`) and + `CwlPdXrayInstrument` (tag `cwl-pd-xray`). Give **both** the full + Bragg-only `Compatibility`: + `sample_form=frozenset({SampleFormEnum.POWDER})`, + `scattering_type=frozenset({ScatteringTypeEnum.BRAGG})`, + `beam_mode=frozenset({BeamModeEnum.CONSTANT_WAVELENGTH})`, and + `radiation_probe=frozenset({RadiationProbeEnum.NEUTRON})` / + `{RadiationProbeEnum.XRAY})` respectively; and + `calculator_support=CalculatorSupport(calculators=frozenset( {CalculatorEnum.CRYSPY, CalculatorEnum.CRYSFML}))` + — **no `PDFFIT`, no `TOTAL`**. On the X-ray class declare the two + `NumericDescriptor`s (with `DisplayHandler`, `AttributeSpec` + defaults/validators, and `TagSpec` exactly as ADR Decision 3) plus + their getter/setter properties (mirror the `setup_wavelength_2` + property style). In `factory.py` replace the `cwl-pd` rule with + two rules keyed on `radiation_probe`. In `bragg_pd.py` pass + `radiation_probe=...` to `default_tag(...)`. Update + `instrument/__init__.py` registration. Leave the data-range + `cwl-pd` tag untouched. Commit: + `Split CW powder instrument by radiation probe` + +- [x] **P1.3 — Add the shared Lorentz-polarization helper.** New module + `analysis/calculators/polarization.py` mirroring `absorption.py`: + `monochromator_cthm(twotheta_deg) -> float` + (`cos²(radians(twotheta))`), + `lp_factor(two_theta, k, cthm) -> np.ndarray` + (`1 - k + k*cthm*cos²(two_theta)`), and + `apply(y, experiment) -> np.ndarray` that no-ops unless the + instrument exposes `setup_polarization_coefficient` with a + non-zero value, otherwise multiplies `y` by `lp_factor` over the + experiment's 2θ grid. Commit: + `Add shared Lorentz-polarization helper` + +- [x] **P1.4 — Bind cryspy natively.** In `cryspy.py` + `_cif_instrument_section`, within the existing powder branch, when + `hasattr(instrument, 'setup_polarization_coefficient')` append + `_setup_K ` and + `_setup_cthm ` + (import `monochromator_cthm` from P1.3). Do **not** apply + `polarization.apply` to cryspy output. If the cached working dict + in `_update_experiment_in_cryspy_dict` exposes the keys, patch + them with a presence guard (mirroring `offset_sycos`); otherwise + document reliance on the CIF-build path + cache rebuild (open + question 2). Commit: + `Emit cryspy polarization setup for X-ray CW powder` + +- [x] **P1.5 — Bind crysfml (verify native line first, then fall + back).** Satisfy the ADR Decision 5 gate explicitly: statically + inspect the active CrysFML CFL grammar / bundled examples for a + native polarization or monochromator line. **If one exists**, emit + that native line and update the ADR/plan to record it (or stop and + ask if it reopens scope). **If none exists** (the expected + outcome), apply `polarization.apply(y, experiment)` on the crysfml + return path alongside the existing + `absorption_correction.apply(...)` (~`crysfml.py:170`); no-op for + neutron / zero coefficient. State the inspection result in the + commit body. Commit: + `Apply Lorentz-polarization envelope on crysfml CW powder` + +- [x] **P1.6 — Demonstrate the new fields in docs.** Per open question + 1: either add the `pd-xray-pbso4-single-polarized-wdt48.py` + verification source (if the FullProf reference data is available) + or set `setup_polarization_coefficient` / + `setup_monochromator_twotheta` in an existing X-ray CW source and + show the pattern responds. Edit the `.py` source only, then run + `pixi run notebook-prepare` and commit the source plus regenerated + notebook. Commit: `Document X-ray CW polarization optics fields` - [x] **P1.7 — Phase 1 review gate.** No-code step. Mark complete and - commit the checklist update alone. - Commit: `Reach Phase 1 review gate` + commit the checklist update alone. Commit: + `Reach Phase 1 review gate` ## Phase 2 — Verification @@ -226,8 +231,8 @@ zsh-safe pattern where output is needed for analysis. Tests to add/update: - `test_factory.py` — `default_tag(..., radiation_probe=NEUTRON|XRAY)` - resolves to `cwl-pd-neutron` / `cwl-pd-xray`; `create()` for both tags; - remove `'cwl-pd'` instrument assertions. + resolves to `cwl-pd-neutron` / `cwl-pd-xray`; `create()` for both + tags; remove `'cwl-pd'` instrument assertions. - `test_support.py` — replace the `cwl-pd` calculator-support assertion with the two new tags, asserting **only** the Bragg engines (`{CRYSPY, CRYSFML}`) for `cwl-pd-neutron` / `cwl-pd-xray` (no diff --git a/docs/docs/tutorials/simulate-nacl-xray.ipynb b/docs/docs/tutorials/simulate-nacl-xray.ipynb index 6ae11ab4c..d6fc42df9 100644 --- a/docs/docs/tutorials/simulate-nacl-xray.ipynb +++ b/docs/docs/tutorials/simulate-nacl-xray.ipynb @@ -51,8 +51,9 @@ "metadata": {}, "outputs": [], "source": [ - "import easydiffraction as edi\n", - "import numpy as np" + "import numpy as np\n", + "\n", + "import easydiffraction as edi" ] }, { diff --git a/docs/docs/tutorials/simulate-nacl-xray.py b/docs/docs/tutorials/simulate-nacl-xray.py index dd880d1b1..4e089d466 100644 --- a/docs/docs/tutorials/simulate-nacl-xray.py +++ b/docs/docs/tutorials/simulate-nacl-xray.py @@ -14,9 +14,10 @@ # ## 🛠️ Import Library # %% -import easydiffraction as edi import numpy as np +import easydiffraction as edi + # %% [markdown] # ## 📦 Define Project diff --git a/src/easydiffraction/analysis/calculators/cryspy.py b/src/easydiffraction/analysis/calculators/cryspy.py index 5a0b1e67b..5e8f63520 100644 --- a/src/easydiffraction/analysis/calculators/cryspy.py +++ b/src/easydiffraction/analysis/calculators/cryspy.py @@ -1182,8 +1182,7 @@ def _cif_polarization_section( return coefficient, monochromator_twotheta = settings cthm = polarization_correction.monochromator_cthm(monochromator_twotheta) - cif_lines.append(f'_setup_K {coefficient}') - cif_lines.append(f'_setup_cthm {cthm}') + cif_lines.extend((f'_setup_K {coefficient}', f'_setup_cthm {cthm}')) def _polarization_settings(experiment: object) -> tuple[float, float] | None: @@ -1228,7 +1227,9 @@ def _set_cryspy_scalar( key: str, value: float, ) -> None: - """Set a Cryspy scalar stored either directly or in a 1-item array.""" + """ + Set a Cryspy scalar stored either directly or in a 1-item array. + """ target = cryspy_expt_dict[key] if isinstance(target, (np.ndarray, list)): target[0] = value diff --git a/src/easydiffraction/analysis/calculators/polarization.py b/src/easydiffraction/analysis/calculators/polarization.py index b9cb79840..ac3f46a34 100644 --- a/src/easydiffraction/analysis/calculators/polarization.py +++ b/src/easydiffraction/analysis/calculators/polarization.py @@ -4,9 +4,9 @@ Calculator-independent CW Lorentz-polarization correction factor. The public instrument stores the physical monochromator angle in -degrees. Backends that expose the FullProf/Cryspy form consume -``cthm = cos^2(2theta_m)``, while backends without a native field use -the same value in the pointwise multiplier here. +degrees. Backends that expose the FullProf/Cryspy form consume ``cthm = +cos^2(2theta_m)``, while backends without a native field use the same +value in the pointwise multiplier here. """ from __future__ import annotations @@ -62,10 +62,10 @@ def apply(y: object, experiment: object) -> object: """ Apply the CW Lorentz-polarization factor to calculated intensities. - The pattern is returned unchanged unless the experiment has an - X-ray CW powder instrument with a nonzero polarization coefficient. - Empty arrays and backend no-data paths whose shape does not match - the experiment 2-theta grid are also left unchanged. + The pattern is returned unchanged unless the experiment has an X-ray + CW powder instrument with a nonzero polarization coefficient. Empty + arrays and backend no-data paths whose shape does not match the + experiment 2-theta grid are also left unchanged. Parameters ---------- @@ -85,7 +85,7 @@ def apply(y: object, experiment: object) -> object: return y coefficient = instrument.setup_polarization_coefficient.value - if coefficient == 0.0: + if not coefficient: return y y_values = np.asarray(y, dtype=float) diff --git a/tests/unit/easydiffraction/analysis/calculators/test_cryspy.py b/tests/unit/easydiffraction/analysis/calculators/test_cryspy.py index e9aae5e57..1327bcb49 100644 --- a/tests/unit/easydiffraction/analysis/calculators/test_cryspy.py +++ b/tests/unit/easydiffraction/analysis/calculators/test_cryspy.py @@ -92,7 +92,9 @@ def test_tof_pseudo_voigt_cif_section_uses_non_convoluted_peak_shape(): def test_cwl_cif_instrument_section_emits_sycos_sysin(): import easydiffraction.analysis.calculators.cryspy as MUT - from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument + from easydiffraction.datablocks.experiment.categories.instrument.cwl import ( + CwlPdNeutronInstrument, + ) from easydiffraction.datablocks.experiment.item.enums import BeamModeEnum from easydiffraction.datablocks.experiment.item.enums import SampleFormEnum diff --git a/tests/unit/easydiffraction/io/cif/test_serialize.py b/tests/unit/easydiffraction/io/cif/test_serialize.py index 8aba9c215..a1c6bee7e 100644 --- a/tests/unit/easydiffraction/io/cif/test_serialize.py +++ b/tests/unit/easydiffraction/io/cif/test_serialize.py @@ -318,7 +318,9 @@ def test_beta_atom_round_trips_through_cif(): def test_cwl_second_wavelength_round_trips_through_cif(): import gemmi - from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument + from easydiffraction.datablocks.experiment.categories.instrument.cwl import ( + CwlPdNeutronInstrument, + ) instr = CwlPdNeutronInstrument() instr.setup_wavelength = 1.5406 @@ -336,7 +338,9 @@ def test_cwl_second_wavelength_round_trips_through_cif(): def test_cwl_disabled_second_wavelength_preserves_value_through_cif(): import gemmi - from easydiffraction.datablocks.experiment.categories.instrument.cwl import CwlPdNeutronInstrument + from easydiffraction.datablocks.experiment.categories.instrument.cwl import ( + CwlPdNeutronInstrument, + ) # Disabled state (ratio == 0) still persists the recorded λ₂. instr = CwlPdNeutronInstrument() From 90435e82fa348d7a8be56dba8308024293953921 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 01:24:06 +0200 Subject: [PATCH 14/16] Complete Phase 2 verification --- docs/dev/plans/xray-cw-polarization-optics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md index 8f2cbe91a..cb4148cc9 100644 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ b/docs/dev/plans/xray-cw-polarization-optics.md @@ -270,7 +270,7 @@ include them only in the `pixi run fix` commit. Leave generated - [x] P1.5 — Bind crysfml (verify native line first, then fall back) - [x] P1.6 — Demonstrate the new fields in docs - [x] P1.7 — Phase 1 review gate -- [ ] Phase 2 — tests added/updated and all five tasks pass +- [x] Phase 2 — tests added/updated and all five tasks pass ## Suggested Pull Request From 99104889767e17cc79f8b4bcadb93e1d153e6660 Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 08:17:00 +0200 Subject: [PATCH 15/16] Refactor documentation and update FullProf label initialization in LBCO verification scripts --- .../pd-neut-cwl_pv-beba_pbso4.ipynb | 7 ++-- .../verification/pd-neut-cwl_pv-beba_pbso4.py | 7 ++-- .../pd-neut-cwl_pv-march_lbco.ipynb | 32 +++---------------- .../verification/pd-neut-cwl_pv-march_lbco.py | 28 +++------------- .../verification/pd-neut-cwl_pv_lbco.ipynb | 2 +- docs/docs/verification/pd-neut-cwl_pv_lbco.py | 2 +- 6 files changed, 15 insertions(+), 63 deletions(-) diff --git a/docs/docs/verification/pd-neut-cwl_pv-beba_pbso4.ipynb b/docs/docs/verification/pd-neut-cwl_pv-beba_pbso4.ipynb index 9bba4956d..a17d96a0c 100644 --- a/docs/docs/verification/pd-neut-cwl_pv-beba_pbso4.ipynb +++ b/docs/docs/verification/pd-neut-cwl_pv-beba_pbso4.ipynb @@ -26,8 +26,6 @@ "source": [ "# PbSO₄ — neutron powder, constant wavelength, Bérar–Baldinozzi asymmetry\n", "\n", - "**Note — cryspy vs FullProf.**\n", - "\n", "cryspy and FullProf implement the\n", "Bérar–Baldinozzi empirical asymmetry with different conventions: an\n", "overall sign and a coefficient inside the `F_b` term differ between the\n", @@ -159,8 +157,8 @@ "FULLPROF_PROJECT_DIR = 'pd-neut-cwl_pv-beba_pbso4'\n", "FULLPROF_PRF_FILE = 'pbso4.prf'\n", "FULLPROF_SUM_FILE = 'pbso4.sum'\n", - "FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE)\n", "FULLPROF_BAC_FILE = 'pbso4.bac'\n", + "FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE)\n", "FULLPROF_ZERO = -0.08424 # FullProf Zero\n", "FULLPROF_SCALE = 1.463815 # FullProf Scale\n", "FULLPROF_WAVELENGTH = 1.912000 # FullProf Lambda\n", @@ -217,8 +215,6 @@ "experiment.peak.broad_gauss_w = FULLPROF_W\n", "experiment.peak.broad_lorentz_x = FULLPROF_X\n", "experiment.peak.broad_lorentz_y = FULLPROF_Y\n", - "# The Berar-Baldinozzi asymmetry coefficients (cryspy only) are set in\n", - "# the cryspy section below; crysfml has no empirical-asymmetry model.\n", "\n", "project.experiments.add(experiment)" ] @@ -244,6 +240,7 @@ "experiment.peak.broad_gauss_w = FULLPROF_W\n", "experiment.peak.broad_lorentz_x = FULLPROF_X\n", "experiment.peak.broad_lorentz_y = FULLPROF_Y\n", + "# crysfml has no Berar-Baldinozzi empirical asymmetry model.\n", "experiment.peak.asym_beba_a0 = FULLPROF_ASY_1\n", "experiment.peak.asym_beba_b0 = FULLPROF_ASY_2\n", "experiment.peak.asym_beba_a1 = FULLPROF_ASY_3\n", diff --git a/docs/docs/verification/pd-neut-cwl_pv-beba_pbso4.py b/docs/docs/verification/pd-neut-cwl_pv-beba_pbso4.py index 94f325928..53e0dab9f 100644 --- a/docs/docs/verification/pd-neut-cwl_pv-beba_pbso4.py +++ b/docs/docs/verification/pd-neut-cwl_pv-beba_pbso4.py @@ -1,8 +1,6 @@ # %% [markdown] # # PbSO₄ — neutron powder, constant wavelength, Bérar–Baldinozzi asymmetry # -# **Note — cryspy vs FullProf.** -# # cryspy and FullProf implement the # Bérar–Baldinozzi empirical asymmetry with different conventions: an # overall sign and a coefficient inside the `F_b` term differ between the @@ -91,8 +89,8 @@ FULLPROF_PROJECT_DIR = 'pd-neut-cwl_pv-beba_pbso4' FULLPROF_PRF_FILE = 'pbso4.prf' FULLPROF_SUM_FILE = 'pbso4.sum' -FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE) FULLPROF_BAC_FILE = 'pbso4.bac' +FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE) FULLPROF_ZERO = -0.08424 # FullProf Zero FULLPROF_SCALE = 1.463815 # FullProf Scale FULLPROF_WAVELENGTH = 1.912000 # FullProf Lambda @@ -137,8 +135,6 @@ experiment.peak.broad_gauss_w = FULLPROF_W experiment.peak.broad_lorentz_x = FULLPROF_X experiment.peak.broad_lorentz_y = FULLPROF_Y -# The Berar-Baldinozzi asymmetry coefficients (cryspy only) are set in -# the cryspy section below; crysfml has no empirical-asymmetry model. project.experiments.add(experiment) @@ -152,6 +148,7 @@ experiment.peak.broad_gauss_w = FULLPROF_W experiment.peak.broad_lorentz_x = FULLPROF_X experiment.peak.broad_lorentz_y = FULLPROF_Y +# crysfml has no Berar-Baldinozzi empirical asymmetry model. experiment.peak.asym_beba_a0 = FULLPROF_ASY_1 experiment.peak.asym_beba_b0 = FULLPROF_ASY_2 experiment.peak.asym_beba_a1 = FULLPROF_ASY_3 diff --git a/docs/docs/verification/pd-neut-cwl_pv-march_lbco.ipynb b/docs/docs/verification/pd-neut-cwl_pv-march_lbco.ipynb index b4217bacf..45759eac9 100644 --- a/docs/docs/verification/pd-neut-cwl_pv-march_lbco.ipynb +++ b/docs/docs/verification/pd-neut-cwl_pv-march_lbco.ipynb @@ -26,8 +26,7 @@ "source": [ "# LBCO — preferred orientation (March–Dollase)\n", "\n", - "Cross-engine check of the **two-parameter** March–Dollase preferred-\n", - "orientation correction. FullProf applies the standard model (its\n", + "Cross-engine check of the **two-parameter** March–Dollase preferred-orientation correction. FullProf applies the standard model (its\n", "`.out` reports \"March-Dollase model for preferred orientation\") with\n", "`Pref1 = 1.2` and `Pref2 = 0.3` along `[0 0 1]`.\n", "\n", @@ -37,9 +36,7 @@ "overall factor that is not volume-normalised; the backend inverts\n", "`march_r` automatically, and the constant non-normalisation factor is\n", "absorbed by the scale (so the as-calculated pattern below shows an\n", - "overall offset before fitting). After refining the two March–Dollase\n", - "parameters and the scale, edi-cryspy recovers `march_r ≈ 1.2` and\n", - "`march_random_fract ≈ 0.3` and the patterns agree." + "overall offset before fitting). After refining the scale, the patterns agree." ] }, { @@ -156,8 +153,8 @@ "FULLPROF_PROJECT_DIR = 'pd-neut-cwl_pv-march_lbco'\n", "FULLPROF_PRF_FILE = 'lbco.prf'\n", "FULLPROF_SUM_FILE = 'lbco.sum'\n", - "FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE)\n", "FULLPROF_BAC_FILE = 'lbco.bac'\n", + "FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE)\n", "FULLPROF_ZERO = 0.62040 # FullProf Zero\n", "FULLPROF_SCALE = 9.405870 # FullProf Scale\n", "FULLPROF_WAVELENGTH = 1.494000 # FullProf Lambda\n", @@ -263,12 +260,7 @@ "id": "13", "metadata": {}, "source": [ - "## Fit edi-cryspy to FullProf\n", - "\n", - "Free the two March–Dollase parameters (`march_r`, `march_random_fract`)\n", - "and the scale, then refine. Starting from the FullProf values,\n", - "edi-cryspy converges back to `march_r ≈ Pref1` and\n", - "`march_random_fract ≈ Pref2`, and the patterns agree." + "## Fit edi-cryspy to FullProf" ] }, { @@ -279,8 +271,6 @@ "outputs": [], "source": [ "experiment.linked_structures['lbco'].scale.free = True\n", - "experiment.preferred_orientation['lbco'].march_r.free = True\n", - "experiment.preferred_orientation['lbco'].march_random_fract.free = True\n", "\n", "project.analysis.fit()\n", "project.display.fit.results()\n", @@ -319,20 +309,6 @@ " ],\n", ")" ] - }, - { - "cell_type": "markdown", - "id": "17", - "metadata": {}, - "source": [ - "The refined `march_r` returns to the FullProf `Pref1 = 1.2` and\n", - "`march_random_fract` to `Pref2 = 0.3` (the latter only approximately,\n", - "because CrysPy's non-normalised factor makes the random-fraction\n", - "correspondence slightly non-linear), with all closeness metrics within\n", - "tolerance. edi-cryspy therefore reproduces the FullProf two-parameter\n", - "March–Dollase correction once its reciprocal/unnormalised convention is\n", - "accounted for by the backend mapping and the scale." - ] } ], "metadata": { diff --git a/docs/docs/verification/pd-neut-cwl_pv-march_lbco.py b/docs/docs/verification/pd-neut-cwl_pv-march_lbco.py index 63e69affa..ffbe5550a 100644 --- a/docs/docs/verification/pd-neut-cwl_pv-march_lbco.py +++ b/docs/docs/verification/pd-neut-cwl_pv-march_lbco.py @@ -1,9 +1,9 @@ # %% [markdown] # # LBCO — preferred orientation (March–Dollase) # -# Cross-engine check of the **two-parameter** March–Dollase preferred- -# orientation correction. FullProf applies the standard model (its -# `.out` reports "March-Dollase model for preferred orientation") with +# Cross-engine check of the **two-parameter** March-Dollase +# preferred-orientation correction. FullProf applies the standard model +# (its `.out` reports "March-Dollase model for preferred orientation") with # `Pref1 = 1.2` and `Pref2 = 0.3` along `[0 0 1]`. # # EasyDiffraction exposes the standard `march_r` (= `Pref1`) and @@ -12,9 +12,7 @@ # overall factor that is not volume-normalised; the backend inverts # `march_r` automatically, and the constant non-normalisation factor is # absorbed by the scale (so the as-calculated pattern below shows an -# overall offset before fitting). After refining the two March–Dollase -# parameters and the scale, edi-cryspy recovers `march_r ≈ 1.2` and -# `march_random_fract ≈ 0.3` and the patterns agree. +# overall offset before fitting). After refining the scale, the patterns agree. # %% import easydiffraction as edi @@ -88,8 +86,8 @@ FULLPROF_PROJECT_DIR = 'pd-neut-cwl_pv-march_lbco' FULLPROF_PRF_FILE = 'lbco.prf' FULLPROF_SUM_FILE = 'lbco.sum' -FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE) FULLPROF_BAC_FILE = 'lbco.bac' +FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE) FULLPROF_ZERO = 0.62040 # FullProf Zero FULLPROF_SCALE = 9.405870 # FullProf Scale FULLPROF_WAVELENGTH = 1.494000 # FullProf Lambda @@ -167,16 +165,9 @@ # %% [markdown] # ## Fit edi-cryspy to FullProf -# -# Free the two March–Dollase parameters (`march_r`, `march_random_fract`) -# and the scale, then refine. Starting from the FullProf values, -# edi-cryspy converges back to `march_r ≈ Pref1` and -# `march_random_fract ≈ Pref2`, and the patterns agree. # %% experiment.linked_structures['lbco'].scale.free = True -experiment.preferred_orientation['lbco'].march_r.free = True -experiment.preferred_orientation['lbco'].march_random_fract.free = True project.analysis.fit() project.display.fit.results() @@ -202,12 +193,3 @@ (f'{LABEL_ED_CRYSPY_REFINED} vs {FULLPROF_LABEL}', calc_fullprof, calc_ed_cryspy_refined), ], ) - -# %% [markdown] -# The refined `march_r` returns to the FullProf `Pref1 = 1.2` and -# `march_random_fract` to `Pref2 = 0.3` (the latter only approximately, -# because CrysPy's non-normalised factor makes the random-fraction -# correspondence slightly non-linear), with all closeness metrics within -# tolerance. edi-cryspy therefore reproduces the FullProf two-parameter -# March–Dollase correction once its reciprocal/unnormalised convention is -# accounted for by the backend mapping and the scale. diff --git a/docs/docs/verification/pd-neut-cwl_pv_lbco.ipynb b/docs/docs/verification/pd-neut-cwl_pv_lbco.ipynb index 646e68584..82e658272 100644 --- a/docs/docs/verification/pd-neut-cwl_pv_lbco.ipynb +++ b/docs/docs/verification/pd-neut-cwl_pv_lbco.ipynb @@ -141,8 +141,8 @@ "FULLPROF_PROJECT_DIR = 'pd-neut-cwl_pv_lbco'\n", "FULLPROF_PRF_FILE = 'lbco.prf'\n", "FULLPROF_SUM_FILE = 'lbco.sum'\n", - "FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE)\n", "FULLPROF_BAC_FILE = 'lbco.bac'\n", + "FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE)\n", "FULLPROF_ZERO = 0.62040 # FullProf Zero\n", "FULLPROF_SCALE = 9.405870 # FullProf Scale\n", "FULLPROF_WAVELENGTH = 1.494000 # FullProf Lambda\n", diff --git a/docs/docs/verification/pd-neut-cwl_pv_lbco.py b/docs/docs/verification/pd-neut-cwl_pv_lbco.py index 965a210f5..f858a965d 100644 --- a/docs/docs/verification/pd-neut-cwl_pv_lbco.py +++ b/docs/docs/verification/pd-neut-cwl_pv_lbco.py @@ -73,8 +73,8 @@ FULLPROF_PROJECT_DIR = 'pd-neut-cwl_pv_lbco' FULLPROF_PRF_FILE = 'lbco.prf' FULLPROF_SUM_FILE = 'lbco.sum' -FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE) FULLPROF_BAC_FILE = 'lbco.bac' +FULLPROF_LABEL = verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_SUM_FILE) FULLPROF_ZERO = 0.62040 # FullProf Zero FULLPROF_SCALE = 9.405870 # FullProf Scale FULLPROF_WAVELENGTH = 1.494000 # FullProf Lambda From c95c584a02d8c5a2e9ddad538c6ac4f1726a9fea Mon Sep 17 00:00:00 2001 From: Andrew Sazonov Date: Thu, 18 Jun 2026 08:28:21 +0200 Subject: [PATCH 16/16] Remove implemented feature plans --- .../cwl-second-wavelength-placeholder.md | 320 ------------------ docs/dev/plans/dataset-driven-fit-modes.md | 292 ---------------- .../dev/plans/verification-regression-flag.md | 310 ----------------- .../verification-software-version-labels.md | 282 --------------- docs/dev/plans/xray-cw-polarization-optics.md | 288 ---------------- 5 files changed, 1492 deletions(-) delete mode 100644 docs/dev/plans/cwl-second-wavelength-placeholder.md delete mode 100644 docs/dev/plans/dataset-driven-fit-modes.md delete mode 100644 docs/dev/plans/verification-regression-flag.md delete mode 100644 docs/dev/plans/verification-software-version-labels.md delete mode 100644 docs/dev/plans/xray-cw-polarization-optics.md diff --git a/docs/dev/plans/cwl-second-wavelength-placeholder.md b/docs/dev/plans/cwl-second-wavelength-placeholder.md deleted file mode 100644 index 58808beb5..000000000 --- a/docs/dev/plans/cwl-second-wavelength-placeholder.md +++ /dev/null @@ -1,320 +0,0 @@ -# Plan: Second-Wavelength (Kα₁/Kα₂) Placeholder on the CWL Instrument - -Follows [`AGENTS.md`](../../../AGENTS.md). No deliberate exceptions to -those instructions. - -## ADR - -No new ADR is required. This adds two parameters to an existing -`CategoryItem` (the constant-wavelength instrument) and serialises them -through the established CIF machinery; it introduces no new category, -factory, switchable-category wiring, or datablock. It touches CIF -serialisation only by populating the **already-scaffolded** -`_diffrn_radiation_wavelength` loop, so it stays within -[`edstar-project-persistence`](../adrs/accepted/edstar-project-persistence.md) -(which documents the existing `_instrument.setup_wavelength` ↔ -`_diffrn_radiation_wavelength.value` mapping). If review decides the -collision/precedence rules below deserve a recorded decision, promote -this section to a short ADR before the PR. - -That ADR carries a persisted-field **inventory** (its CWL-instrument -rows and the per-attribute Edi-name table), so adding two persisted -fields makes the ADR stale unless it is updated in lockstep. This plan -therefore includes a Phase 1 step (P1.3) to update those inventory rows -— it amends the inventory, not the decision (Review 1, F3). - -## Summary of the change - -Constant-wavelength instruments today carry a single `setup_wavelength`. -Laboratory X-ray sources (and the FullProf X-ray reference under -`docs/docs/verification/fullprof/pd-xray-pbso4/`) use a Cu Kα₁/Kα₂ -**doublet**: a second wavelength with a fixed relative intensity. -FullProf encodes this as `Lambda1 Lambda2 Ratio`; the future crysfml CFL -reads it as `LAMBDA λ₁ λ₂ ratio`; CIF core encodes it as a looped -`DIFFRN_RADIATION_WAVELENGTH` with per-row `value` + `wt`. - -This change adds a **placeholder** for that doublet on the CWL -instrument category: two new parameters that can be set, read, and -round-trip through CIF. **Binding to the calculation engines is out of -scope** — no calculator reads these values yet, so a doublet experiment -calculates exactly as today (single wavelength). cryspy has no -second-wavelength support and is explicitly not addressed; the eventual -consumer is a future crysfml build via the `LAMBDA` CFL directive. - -New public settings on `CwlInstrumentBase`, both **non-refinable** -`NumericDescriptor`s (see Decision "Non-refinable by construction"): - -- `setup_wavelength_2` — the second wavelength λ₂ (Å). Default `0.0` - meaning "no second component" (monochromatic, today's behaviour). -- `setup_wavelength_2_to_1_ratio` — the relative intensity of the second - component to the first, **I(wavelength_2) / I(wavelength)**. The - `_2_to_1_` ordering names the direction explicitly (numerator = - component 2, denominator = component 1). Default `0.0` (second - component contributes nothing). This is the FullProf `Ratio` column, - the CFL `LAMBDA` third value, and the CIF `wt` of the second loop row - (with the first row's `wt` normalised to 1). - -### Reachable states (all four are publicly settable) - -The two fields are set independently, so every combination is reachable. -The semantics are fully specified — there are no undefined mixed states: - -| `setup_wavelength_2` | `…_2_to_1_ratio` | Meaning | CIF / output | -| -------------------- | ---------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------- | -| `0` | `0` | Monochromatic (default, today) | single row / scalar | -| `> 0` | `0` | Second λ recorded but **disabled** — matches the CFL `LAMBDA … 0.0` convention | single row / scalar; λ₂ value preserved on edi-CIF round-trip | -| `> 0` | `> 0` | **Active doublet** | 2-row `_diffrn_radiation_wavelength` loop | -| `0` | `> 0` | **Invalid**: a relative intensity for a wavelength that does not exist | export raises a clear `ValueError` | - -So "active doublet" ≡ -`setup_wavelength_2 > 0 and setup_wavelength_2_to_1_ratio > 0`; -"disabled" is governed by `ratio == 0` (any λ₂); the incomplete pair -`(λ₂ == 0, ratio > 0)` is a boundary user-input error and is rejected, -not silently dropped. - -## Decisions - -- **Names.** `setup_wavelength_2` and `setup_wavelength_2_to_1_ratio`, - chosen in conversation. The ratio name encodes numerator/denominator - direction so it cannot be misread as `λ/λ₂` vs `λ₂/λ`. -- **Scope.** Settings **plus CIF round-trip**; no engine binding. -- **Non-refinable by construction (Review 1, F1).** Both fields are - `NumericDescriptor`s, **not** `Parameter`s. A `NumericDescriptor` has - no `free` flag, so the fitter — which collects only free `Parameter`s - (`src/easydiffraction/analysis/fitting.py:103`) and syncs them back - each residual evaluation (`…/fitting.py:479`) — can never pick these - up. This removes the silent no-op a refinable but engine-unbound value - would create: a scientist cannot mark a doublet setting `free` and - watch the optimiser move a value that changes nothing. They follow the - existing editable-but-non-refinable pattern used for the CWL - data-range bounds (`…/categories/data_range/cwl.py`). When engine - binding lands, a follow-up can promote them to `Parameter`. -- **Mixed states are fully specified (Review 1, F2).** See the - "Reachable states" table above. `ratio == 0` means _disabled_ - (single-row output, λ₂ preserved), matching the CFL `LAMBDA … 0.0` - convention; `ratio > 0 and λ₂ == 0` is an incomplete pair and is - **rejected at export time** with a clear `ValueError` (via the - project's `log.error(..., exc_type=ValueError)` pattern) rather than - silently emitting a single row and dropping the ratio. -- **Ratio is a relative intensity, not a wavelength ratio.** Range - `[0, 1]` (matches CIF `wt` `_enumeration.range 0.0:1.0`). Documented - in the docstring and field description. -- **edi-CIF round-trip is automatic.** `category_item_to_cif` writes - each field by its **`edi_name`** (`io/cif/serialize.py:189`), and - `category_item_from_cif` reads by the union `read_names` - (`io/cif/handler.py` `read_names`); `NumericDescriptor`s serialise - through this same path (the CWL data-range bounds prove it). Distinct - `edi_names` (`_instrument.setup_wavelength_2`, - `_instrument.setup_wavelength_2_to_1_ratio`) give a clean scalar - round-trip with **no extra serialise code** — the placeholder persists - the moment the fields exist. -- **IUCr/pdCIF export uses the loop.** The strict export - (`io/cif/iucr_writer.py` → `WavelengthTransformer`) emits the proper - `_diffrn_radiation_wavelength` loop when the doublet is active and - keeps today's single-row scalar output otherwise. The transformer - already has an `items()` path and an unused `loop()` stub - (`io/cif/iucr_transformers.py:107`) reserved for exactly this. -- **Avoid the shared-tag collision.** `setup_wavelength` already lists - `_diffrn_radiation_wavelength.value` in its `cif_names`. The two new - fields' `cif_names` contain **only** their edi-CIF short names - (`_instr.wavelength_2`, `_instr.wavelength_2_to_1_ratio`) — they do - **not** list any `_diffrn_radiation_wavelength.*` tag, so the generic - scalar writer/reader can never emit or consume a duplicate bare - `_diffrn_radiation_wavelength.value`/`.wt`. The IUCr `value`/`wt` - pairing is produced solely by the loop transformer, which - disambiguates components by row `id`. (`setup_wavelength`'s existing - tags are left unchanged.) - -## Open questions - -1. **IUCr loop _import_.** This plan round-trips through **edi-CIF** - (the project-persistence format) and additionally _exports_ the IUCr - loop. Reading a 2-row `_diffrn_radiation_wavelength` loop back from a - strict-IUCr file into the two parameters is **not** in scope — the - generic scalar reader only sees the first row. Flag for a follow-up - if strict-IUCr import of doublets is wanted. (Recommended: defer.) -2. **`xray_symbol` / `type` metadata.** CIF core also offers - `_diffrn_radiation_wavelength.xray_symbol` (e.g. `K-L~3~`/`K-L~2~`) - and `.type`. Out of scope for the placeholder; note as deferred. -3. **TOF instruments.** The doublet is a CWL concept; no change to - `instrument/tof.py`. Confirm reviewers agree the fields live on - `CwlInstrumentBase` only. -4. **Where the incomplete-pair guard lives.** The plan raises the - `(λ₂ == 0, ratio > 0)` error at IUCr export (in the transformer), - because the two fields are set independently and a set-time check - would trip on the transient half-set state. Reviewers may prefer an - additional validation hook; export-time is proposed as the single - enforced boundary for the placeholder. (Recommended: export-time - only.) - -## Deferred work — engine binding and its performance note - -Engine binding is out of scope here, but recording the trade-off so the -follow-up starts informed: - -- **Two ways to compute the doublet.** (a) _Calculator-independent_ — - edi calls the engine once at λ₁ and once at λ₂ and weight-sums the two - patterns; works for any engine (and is the only option for cryspy, - which has no native doublet). (b) _Native crysfml_ — a future build - computes both lines in one pass via the `LAMBDA λ₁ λ₂ ratio` CFL - directive. -- **(a) is expected to be slower, bounded by ~2× the single-wavelength - cost.** In crysfml's `cw_powder_pattern_profile` - (`tmp/crysfml/Src/CFML_Utilities/Utilities_Patterns.f90`) the - per-reflection structure factors `ref(i)%fc(2)**2` are computed - **once** before the reflection loop; a native doublet extends that - same loop to lay down a second peak at the λ₂ position scaled by - `ratio`, reusing the same |F|². So native pays 1× structure-factors + - 1× reflection generation + ~2× profile summation + 1× call overhead, - whereas the two-pass approach repeats **all** of that. The gap is - small when profile summation dominates (fine grid / broad peaks) and - approaches the full 2× when structure-factor / reflection-generation / - marshalling dominate; it compounds across a fit's many residual - evaluations. -- **Caveats.** The crysfml routine wired into easydiffraction today is - itself single-wavelength (uses `Lambda(1)`, ignores the `twowaves` - flag set in `Format_CFL.f90`), so even the native path needs the - future build. Because Kα₁/Kα₂ differ by ~0.25%, |F|² is effectively - identical for both lines — so a smarter calculator-independent variant - could compute reflections/|F|² once and place only the second peak set - itself, matching native's "compute once, place twice", at the cost of - edi owning profile generation (a larger change than this placeholder). - -## Concrete files likely to change - -- `src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py` - — add the two `NumericDescriptor`s on `CwlInstrumentBase` with - getter/setter properties (mirroring the `data_range/cwl.py` - non-refinable pattern), distinct `edi_names`, range `ge=0.0` / - `[0, 1]` validators, and `TagSpec`s carrying only the `_instr.*` short - `cif_names` per the collision decision. -- `src/easydiffraction/io/cif/iucr_transformers.py` — make - `WavelengthTransformer.items()` return the single row when - monochromatic-or-disabled (`ratio == 0`) and `None` when the doublet - is active (so the writer falls through to `loop()`); implement - `loop()` to emit the 2-row `_diffrn_radiation_wavelength` loop (`id`, - `value`, `wt`) when active; and raise a clear `ValueError` for the - incomplete pair `(λ₂ == 0, ratio > 0)`. -- `src/easydiffraction/io/cif/iucr_writer.py` — - `_write_wavelength_section` already prefers `items()` then `loop()` - (lines 247-269); verify it needs no change once `items()`/`loop()` - cooperate. Adjust only if the fall-through guard - (`if wavelength is None`) interferes. -- `docs/dev/adrs/accepted/edstar-project-persistence.md` — extend the - CWL-instrument inventory row (~line 633) and the per-attribute Edi - name table (~line 847) with the two new fields and their `_instr.*` / - `_instrument.*` names and the IUCr loop note (F3). -- Tests (Phase 2): - - `tests/unit/easydiffraction/datablocks/experiment/categories/instrument/test_cwl.py` - — defaults are off; set/get; range validation; monochromatic - behaviour unchanged; **non-refinable** (the field is a - `NumericDescriptor` with no `free`, so it is never collected by the - fitter). - - `tests/unit/easydiffraction/io/cif/test_iucr_transformers.py` and - `test_iucr_writer.py` — scalar output when monochromatic and when - disabled (`ratio == 0`, `λ₂ > 0`); 2-row loop when active; clear - `ValueError` for the incomplete pair `(λ₂ == 0, ratio > 0)`. - - edi-CIF round-trip: a `setup_wavelength_2`/ratio set on an - experiment (including the disabled `ratio == 0`, `λ₂ > 0` state) - survives `as_cif` → `from_cif` (extend the nearest existing - serialize round-trip test under - `tests/unit/easydiffraction/io/cif/`). -- Docs: no tutorial change. The `docs/docs/verification/pd-xray-pbso4` - page stays a `known_discrepancy` (engine binding is still absent), so - it is untouched by this plan. - -## Branch and PR notes - -- Flat-slug implementation branch off `develop`: - `cwl-second-wavelength-placeholder` (created by `/draft-impl-1`, not - now). PR targets `develop`. -- Each Phase 1 step is staged with explicit paths and committed locally - before the next step (per `AGENTS.md` §Commits). Atomic, - single-purpose commits. - -## Implementation steps (Phase 1) - -- [x] **P1.1 — Add the two placeholder fields to `CwlInstrumentBase`.** - In `instrument/cwl.py`, add `_setup_wavelength_2` - (`NumericDescriptor`, default `0.0`, `RangeValidator(ge=0.0)`, - units `angstroms`) and `_setup_wavelength_2_to_1_ratio` - (`NumericDescriptor`, default `0.0`, - `RangeValidator(ge=0.0, le=1.0)`, dimensionless) with getter - properties and value setters following the non-refinable - `data_range/cwl.py` pattern. Give them distinct `edi_names` - (`_instrument.setup_wavelength_2`, - `_instrument.setup_wavelength_2_to_1_ratio`) and `cif_names` - containing only the `_instr.*` short names (collision decision - above). Docstrings state the ratio direction and the `[0, 1]` - range. No tests yet (Phase 1 is code + docstrings only). Commit: - `Add second-wavelength placeholder fields to CWL instrument` - -- [x] **P1.2 — Emit the IUCr wavelength loop and guard mixed states.** - In `iucr_transformers.py`, update `WavelengthTransformer.items()` - to return the single-row items when monochromatic **or disabled** - (`ratio == 0`, any λ₂) and `None` when the doublet is active; - implement `loop()` to return an `IucrLoop` of - `_diffrn_radiation_wavelength.{id,value,wt}` with rows - `(1, λ₁, 1.0)` and `(2, λ₂, ratio)` when active; and raise a clear - `ValueError` (project `log.error(..., exc_type=ValueError)` - pattern) for the incomplete pair `(λ₂ == 0, ratio > 0)`. Verify - `_write_wavelength_section` in `iucr_writer.py` routes correctly; - adjust the guard only if required. Commit: - `Emit diffrn_radiation_wavelength loop for CWL doublet` - -- [x] **P1.3 — Update the Edi persistence inventory ADR.** In - `docs/dev/adrs/accepted/edstar-project-persistence.md`, add the - two new fields to the CWL-instrument inventory row (~line 633) and - the per-attribute Edi-name table (~line 847), with their - `_instr.*` and `_instrument.*` names and an IUCr-loop note. - Inventory amendment only; the decision is unchanged. Commit: - `Record CWL second-wavelength fields in persistence ADR` - -- [x] **P1.4 — Phase 1 review gate.** No-code step. Mark complete and - commit the checklist update alone. Commit: - `Reach Phase 1 review gate` - -## Verification (Phase 2) - -Run in order; capture output with the zsh-safe pattern when analysis is -needed: - -```bash -pixi run fix -pixi run check > /tmp/edi-check.log 2>&1; check_exit_code=$?; tail -n 200 /tmp/edi-check.log; exit $check_exit_code -pixi run unit-tests > /tmp/edi-unit.log 2>&1; unit_tests_exit_code=$?; tail -n 100 /tmp/edi-unit.log; exit $unit_tests_exit_code -pixi run integration-tests -pixi run script-tests -``` - -Phase 2 adds the tests listed under _Concrete files_: - -- CWL instrument: defaults off, set/get, range validation, monochromatic - output unchanged, and **non-refinable** (field is a - `NumericDescriptor` with no `free`; never collected by the fitter). -- IUCr transformer/writer: scalar when monochromatic and when disabled - (`ratio == 0`, `λ₂ > 0`); 2-row loop when active; `ValueError` for the - incomplete pair `(λ₂ == 0, ratio > 0)`. -- edi-CIF round-trip of both new fields, including the disabled state. - -## Status checklist - -- [x] P1.1 — placeholder fields added (non-refinable - `NumericDescriptor`s) -- [x] P1.2 — IUCr loop emitted when active; mixed states guarded -- [x] P1.3 — Edi persistence inventory ADR updated -- [x] P1.4 — Phase 1 review gate -- [ ] Phase 2 — tests added and all five `pixi run` tasks clean - -## Suggested Pull Request - -**Title:** Support a second X-ray wavelength (Kα₁/Kα₂) on -constant-wavelength instruments - -**Description:** Constant-wavelength instruments can now record a second -incident wavelength and its relative intensity — the Kα₁/Kα₂ doublet -typical of laboratory X-ray sources. The new `setup_wavelength_2` and -`setup_wavelength_2_to_1_ratio` settings are saved and restored with the -project and exported in standard CIF. This is a data-model step: the -values are stored and shared but not yet used in pattern calculation, so -existing single-wavelength experiments are unaffected. diff --git a/docs/dev/plans/dataset-driven-fit-modes.md b/docs/dev/plans/dataset-driven-fit-modes.md deleted file mode 100644 index be0fbb952..000000000 --- a/docs/dev/plans/dataset-driven-fit-modes.md +++ /dev/null @@ -1,292 +0,0 @@ -# Plan: Dataset-Driven Fit Mode Availability - -Governed by [`AGENTS.md`](../../../AGENTS.md). No deliberate exceptions -to those instructions are taken in this plan. - -Implements ADR -[`dataset-driven-fit-modes`](../adrs/accepted/dataset-driven-fit-modes.md) -(promoted from a suggestion to `accepted/` in P1.8, per §Change -Discipline). Closes issue **85 — Retain Per-Experiment Fitted Parameters -for Plotting** -([`closed/retain-per-experiment-fitted-parameters-for-plotting.md`](../issues/closed/retain-per-experiment-fitted-parameters-for-plotting.md)) -by removing the `single`-with-N path that caused it. - -## ADR - -This plan **owns** the ADR `dataset-driven-fit-modes`. Phase 1 promotes -it to `accepted/` (P1.8). The change also extends the accepted -[`fit-mode-categories`](../adrs/accepted/fit-mode-categories.md) ADR but -does not modify it. - -## Summary of the change - -- Fit-mode availability becomes **applicability-based**: - `fitting_mode.show_supported()` lists only the modes whose - applicability predicate the loaded project satisfies — `single` - (exactly one loaded experiment), `joint` (≥2), `sequential` (exactly - one loaded experiment). Readiness — each scheduled experiment has - measured data, and `sequential` has a resolvable `data_dir` matching - files — is checked at `fit()`, not for visibility. -- `single` is restricted to exactly one experiment; the multi-experiment - loop (`single`-with-N) is removed, which **closes issue 85** by - construction. -- The now-orphaned `_parameter_snapshots` / - `plot_param_series_from_snapshots` fallback is deleted; - parameter-evolution plotting stays on the `sequential` `results.csv` - path. -- `sequential` keeps its folder-sweep behaviour; its data-source config - gains a clear "no files" fit-time error and a `copy_data` flag - (default off) with a defined, idempotent round-trip contract. (The - template-derived `file_pattern` default from the ADR is deferred — see - Decision 4.) - -## Decisions (from the ADR — see it for rationale) - -1. Applicability drives `show_supported()`; readiness is enforced only - at `fit()`. The `.type` setter stays permissive (any mode), so CIF - restore keeps a stored mode and rejection happens at fit time. -2. **Applicability is by total loaded-experiment count; measured-data is - readiness.** `single`/`sequential` are applicable when the project - holds exactly one loaded experiment, `joint` when it holds ≥2 — - regardless of measured-vs-calculated. The requirement that each - scheduled experiment actually has measured data is a **readiness** - check enforced at `fit()` by the existing - `Fitter._require_measured_data` guard (a calculated-only experiment - yields a clear fit-time error, not a hidden mode). This keeps - `show_supported()` and `fit()` consistent for mixed - measured/calculated projects, and avoids any "schedule only the - measured subset" filtering. **This refines the ADR's "experiment with - measured data" applicability wording into the applicability/readiness - split; the ADR text is aligned at promotion (P1.8).** -3. `copy_data` (default `False`): at `fit()`, before the sweep, copy - matched files into `/data/sequential/`, rewrite persisted - `data_dir` to that relative destination (portability), overwrite on - name conflict; only the `sequential_fit` fields are serialized. The - copy is **idempotent**: when the resolved source directory is already - the copy destination (the post-reload case), the copy is skipped and - the run uses the archived files. -4. `data_dir` has no smart default. **`file_pattern` keeps its current - `'*'` default for the first step**; the ADR's "derive the glob from - the template experiment's data-file extension" is deferred because - the experiment model does not retain its source data-file path today - (deriving it requires new experiment source-path metadata + its - persistence and tests). The ADR's stated `'*'` fallback is therefore - the shipped first-step behaviour; the derived default is a tracked - follow-up. The ADR text is aligned at promotion (P1.8). - -## Open questions - -- **`fitting-exercise-si-lbco.py` pedagogy (P1.7) — RESOLVED.** The - audit in P1.7 found this tutorial uses two separate one-experiment - projects (not multiple experiments in one project), so it does not - rely on `single`-with-N and needs no change. No pedagogy decision is - required. -- **Resume** stays `single`-only as today; per-point `sequential` resume - is out of scope (ADR Open Questions). -- **Template-derived `file_pattern` (deferred).** Needs new experiment - source-data-path metadata (plus persistence and tests). The first step - ships the `'*'` default; a follow-up adds the source-path metadata and - derives `*.ext` from it. Tracked as Deferred Work in the ADR at - promotion (P1.8). - -## Concrete files likely to change - -Phase 1 (implementation): - -- `src/easydiffraction/analysis/categories/fitting_mode/default.py` — - `_supported_types(filters)` returns applicable modes. -- `src/easydiffraction/analysis/analysis.py` — `_supported_filters_for` - context for `fitting_mode` (loaded-experiment count); fit-time - applicability validation in `_validate_fit_request`; collapse - `_fit_single_experiments` to one experiment; remove - `_parameter_snapshots` (line ~602) and `_snapshot_params` (~3009); - sequential data-source resolution + `copy_data` handling; check - `_help_filter` / `_serializable_categories`. -- `src/easydiffraction/analysis/categories/sequential_fit/default.py` — - add `copy_data` `BoolDescriptor` + property (mirror `reverse`). -- `src/easydiffraction/analysis/sequential.py` — no-files error; - idempotent copy (skip self-copy) + `data_dir` rewrite. -- `src/easydiffraction/display/plotting.py` — remove - `plot_param_series_from_snapshots` and the snapshot fallback branches - in `plot_param_series`, `plot_all_param_series`, - `_collect_fitted_parameter_unique_names`; no-CSV path warns and - returns. -- `docs/docs/tutorials/fitting-exercise-si-lbco.py` (+ regenerated - `.ipynb`) — stop relying on `single`-with-N. -- `docs/dev/issues/...` — move issue 85 to `closed/`, update index. -- `docs/dev/adrs/...` — promote ADR to `accepted/`, update index, fix - status/links. - -Phase 2 (verification — tests): - -- `tests/unit/easydiffraction/analysis/categories/test_fitting_mode*` / - `test_fitting*` — applicability predicates given experiment counts. -- `tests/unit/easydiffraction/analysis/test_analysis*` — fit-time - validation errors; single-exactly-one. -- `tests/.../sequential_fit` + - `tests/integration/fitting/test_sequential.py` — `copy_data` - round-trip, idempotent self-copy (source == destination skips), - no-files error. -- `tests/unit/easydiffraction/display/test_plotting_coverage.py`, - `tests/.../test_analysis_coverage.py`, - `tests/integration/fitting/test_analysis_and_fit_category_support.py` - — drop snapshot-based expectations; assert no-CSV warns. - -## Branch and PR notes - -- Branch: `dataset-driven-fit-modes` (already checked out). -- PR targets `develop`. Do not push unless asked. - -## Implementation steps (Phase 1) - -Per §Planning and §Commits: when an AI agent follows this plan, every -completed Phase 1 step is staged with **explicit paths** and committed -locally (atomic, single-purpose) before moving on. Mark each `- [ ]` as -`- [x]` in the same commit that completes it. - -Applicability/readiness contract used across P1.1–P1.3 (Decision 2): -applicability counts **total loaded experiments**; the measured-data -requirement is a **readiness** check left to the existing -`Fitter._require_measured_data` guard at `fit()`. No "schedule only the -measured subset" filtering is introduced. - -- [x] **P1.1 — Applicability-based `show_supported()`.** Add an Analysis - helper that returns the loaded-experiment count - (`len(project.experiments)`); have `_supported_filters_for` pass - that count to the `fitting_mode` category; implement - `FittingMode._supported_types(filters)` to return `single` (count - == 1), `joint` (count ≥ 2), `sequential` (count == 1). Files: - `analysis.py`, `fitting_mode/default.py`. Commit: - `Offer fit modes by project applicability` - -- [x] **P1.2 — Enforce mode preconditions at fit time.** In - `_validate_fit_request`, reject a selected mode whose - applicability predicate the project does not meet — - `single`/`sequential` unless exactly one experiment is loaded, - `joint` unless ≥2 — with a clear `ValueError` naming the valid - modes. Keep the `.type` setter permissive so CIF restore is not - rejected. Measured-data readiness stays with - `Fitter._require_measured_data` (no change needed there). Files: - `analysis.py`. Commit: - `Validate fit mode against loaded experiments` - -- [x] **P1.3 — Restrict `single` to exactly one experiment.** With P1.2 - guaranteeing exactly one loaded experiment for `single`, collapse - `_fit_single_experiments` to fit that one experiment (no loop); - drop the per-experiment `_snapshot_params` call. Files: - `analysis.py`. Commit: `Fit a single experiment in single mode` - -- [x] **P1.4 — Remove the `single`-with-N snapshot machinery.** Delete - `_parameter_snapshots` and `_snapshot_params`; remove - `plot_param_series_from_snapshots` and the snapshot fallback - branches in `plot_param_series`, `plot_all_param_series`, and - `_collect_fitted_parameter_unique_names`; when no `results.csv` - exists, log a clear warning and return. Files: `analysis.py`, - `display/plotting.py`. Commit: - `Remove single-mode parameter snapshot fallback` - -- [x] **P1.5 — Sequential data source: no-files error, `copy_data`.** - Add `copy_data` `BoolDescriptor` (+ property, default `False`) to - `SequentialFit` (mirror `reverse`). In sequential resolution: - raise a clear `ValueError` when `data_dir` is unset/unresolvable - or matches no files (keeping the current `'*'` `file_pattern` - default — the template-extension derivation is deferred, Decision - 4). When `copy_data` is set, apply the **idempotent** copy: if the - resolved source directory is already the copy destination - (`/data/sequential/`, the post-reload case), skip the - copy and run from the archived files; otherwise copy matched files - there (overwrite on name conflict) and rewrite the persisted - `data_dir` to that relative destination. Files: - `sequential_fit/default.py`, `sequential.py`, `analysis.py`. - Commit: `Add copy_data and clearer sequential data resolution` - -- [x] **P1.6 — Reconcile display/serialization filters.** Verify - `_help_filter` and `_serializable_categories` reflect the active - mode under the new model (sequential config hidden in `single`, - etc.); adjust only if needed. Files: `analysis.py`. Commit: - `Align analysis category visibility with fit modes` - -- [x] **P1.7 — Audit tutorials for `single`-with-N (no change needed).** - An audit of every `docs/docs/tutorials/*.py` found **no** tutorial - fits ≥2 experiments in `single`/default mode: - `fitting-exercise-si-lbco.py` uses two _separate_ projects with - one experiment each (`sim_si`, `sim_lbco`) — its repeated `fit()` - calls are progressive single-experiment refinements, not - `single`-with-N; the multi-experiment tutorials - (`calibrate-beer-ess`, `joint-si-bragg-pdf`, `refine-ncaf-wish`) - all set `joint`/`sequential` before fitting. So the earlier - premise that the Si/LBCO exercise relied on `single`-with-N was - incorrect, and no tutorial edit (or `notebook-prepare`) is - required. Commit: `Confirm no tutorial relies on single-with-N` - -- [x] **P1.8 — Close issue 85 and promote the ADR.** `git mv` issue 85 - to - `closed/retain-per-experiment-fitted-parameters-for-plotting.md`, - rewrite its body to describe the resolution (single restricted to - one dataset; snapshot fallback removed), and update - `docs/dev/issues/index.md`. Promote the ADR: `git mv` - `docs/dev/adrs/suggestions/dataset-driven-fit-modes.md` → - `accepted/`, set its `## Status` to `Accepted`, flip its - `docs/dev/adrs/index.md` row to `Accepted` with the `accepted/...` - link, fix any links that pointed at `suggestions/...` - (`git grep -n`), and remove the `_review-*`/`_reply-*` siblings if - still present. **Align the ADR text to the refinements made during - plan review:** Decision 1/applicability wording → - loaded-experiment count with measured-data as readiness (plan - Decision 2); Decision 4 → move the template-derived `file_pattern` - default to Deferred Work and state `'*'` as the shipped default - (plan Decision 4); Decision 3 → note the idempotent self-copy - skip. Files: `docs/dev/issues/...`, `docs/dev/adrs/...`. Commit: - `Close issue 85 and accept dataset-driven fit modes ADR` - -- [x] **P1.9 — Phase 1 review gate.** No-code step. Mark `[x]`, commit - the checklist update alone with message - `Reach Phase 1 review gate`, then stop for Phase 1 review. - -## Verification (Phase 2) - -Add/update tests first (see "Concrete files" Phase 2 list), then run the -full suite. Use the zsh-safe log-capture pattern from §Workflow. - -```bash -pixi run fix -pixi run check > /tmp/easydiffraction-check.log 2>&1; check_exit_code=$?; tail -n 200 /tmp/easydiffraction-check.log; exit $check_exit_code -pixi run unit-tests > /tmp/easydiffraction-unit.log 2>&1; unit_tests_exit_code=$?; tail -n 200 /tmp/easydiffraction-unit.log; exit $unit_tests_exit_code -pixi run integration-tests > /tmp/easydiffraction-integration.log 2>&1; integration_tests_exit_code=$?; tail -n 200 /tmp/easydiffraction-integration.log; exit $integration_tests_exit_code -pixi run script-tests > /tmp/easydiffraction-script.log 2>&1; script_tests_exit_code=$?; tail -n 200 /tmp/easydiffraction-script.log; exit $script_tests_exit_code -pixi run test-structure-check -``` - -Notes: - -- `pixi run fix` regenerates `docs/dev/package-structure/full.md` and - `short.md`; include them only if changed. -- Tutorial project-path collisions, benchmark CSVs, and sandbox-only - multiprocessing failures: handle per §Workflow. - -## Status checklist - -- [x] P1.1 — Applicability-based `show_supported()` -- [x] P1.2 — Enforce mode preconditions at fit time -- [x] P1.3 — Restrict `single` to exactly one experiment -- [x] P1.4 — Remove the `single`-with-N snapshot machinery -- [x] P1.5 — Sequential data source: no-files error, `copy_data` -- [x] P1.6 — Reconcile display/serialization filters -- [x] P1.7 — Update tutorials relying on `single`-with-N -- [x] P1.8 — Close issue 85 and promote the ADR -- [x] P1.9 — Phase 1 review gate -- [x] Phase 2 — tests added and full verification suite green - -## Suggested Pull Request - -**Title:** Clearer fit modes that match your loaded data - -**Description:** EasyDiffraction now offers only the fitting modes that -make sense for what you have loaded: with one dataset you get a normal -single fit (and can switch to a sequential scan over a folder of files); -with several datasets you get a combined (joint) fit. This removes a -confusing case where fitting several datasets one-by-one could make -earlier datasets plot incorrectly. Sequential scans are also clearer to -set up: you get a clear message when no data files are found, and a new -option can copy the scanned files into your project so it stays -self-contained and portable. diff --git a/docs/dev/plans/verification-regression-flag.md b/docs/dev/plans/verification-regression-flag.md deleted file mode 100644 index ee697163f..000000000 --- a/docs/dev/plans/verification-regression-flag.md +++ /dev/null @@ -1,310 +0,0 @@ -# Plan: Notebook-Owned Verification Regression Gating (`known_discrepancy`) - -Follows [`AGENTS.md`](../../../AGENTS.md). Two explicitly-scoped -exceptions to the usual two-phase rules: - -1. A small, unrelated **Windows CI fix** is bundled at the owner's - request (P1.7). -2. **P1.3 runs each verification page once to classify it** - (disagree-cleanly / crash / re-gate) — a deliberate, **read-only** - Phase-1 audit needed to decide each page's migration. It is not test - authoring and runs no test suite; the exact command is in P1.3. All - real test work stays in Phase 2. - -## ADR - -Implements -[`verification-regression-flag`](../adrs/accepted/verification-regression-flag.md) -(Accepted). Per §Change Discipline it is **promoted to `accepted/`** as -part of this PR (P1.6). The sibling -[`verification-software-version-labels`](../adrs/suggestions/verification-software-version-labels.md) -ADR is **not** implemented here — separate work. - -Revises two accepted ADRs (P1.5): -[`test-suite-and-validation`](../adrs/accepted/test-suite-and-validation.md) -§6 (the `script-tests` "skip via `ci_skip.txt`" detail) and -[`documentation-ci-build`](../adrs/accepted/documentation-ci-build.md) -(the `ci_skip.txt`/conftest wiring). - -## Branch + PR - -- Branch: `verification-regression-flag` (flat slug off `develop`, - **already created and checked out**; carries one prior housekeeping - commit, `Remove completed dataset-driven-fit-modes plan`). -- PR targets `develop`, not `master`. Do not push unless asked. - -## Decisions (from the ADR) - -- **Flag.** `assert_patterns_agree` gains - `known_discrepancy: bool = False` (replacing `raise_on_failure`) and a - `reason: str | None = None` (**required** when - `known_discrepancy=True`). -- **Two-sided assertion (Decision 1a).** `known_discrepancy=True` - asserts the patterns **still disagree**: out of tolerance → render, do - not raise; **within** tolerance → **raise** a re-gate - `AssertionError`. -- **Return contract (Decision 1b).** Returns `True` when the page met - its expectation (agree for default; still-disagree for known-bad) or - **raises**. No longer the raw agreement boolean; raw metrics stay on - `verify.pattern_closeness`. -- **One source of truth, two runners (Decision 2).** Delete - `ci_skip.txt`; `script-tests` statically detects - `known_discrepancy=True` **or** a `raises-exception` tag in the `.py` - and **skips** those pages; `conftest.py`'s xfail logic is removed; - **nbmake runs every page** (render + the re-gate hard-fail). -- **Crash boundary (Decision 3).** Pre-flag-crash pages tag the failing - cell `raises-exception` (+ a markdown note); `script-tests` skips them - via the same scan. -- **On-page provenance (Decision 4).** `reason` renders on the page, - replacing the `ci_skip.txt` comment. -- **Fit hygiene (Decision 5).** A page already agreeing within tolerance - carries no fit section. -- **Audit (ADR Open Questions).** Every page using the old - `raise_on_failure=False` is migrated with an explicit per-page - decision; this is a larger set than `ci_skip.txt` (P1.3). - -## Decisions already made for this plan - -- The audit covers **11 pages** — the 10 `.py` files using - `raise_on_failure=False` plus `pd-neut-cwl_tch-fcj-nosldl_lab6` (in - `ci_skip.txt` but not using the flag, so likely a crash page): - - **In `ci_skip.txt` + `raise_on_failure=False`** (7): `beba_pbso4`, - `tch-fcj-noabs-nosldl_lab6`, `tch-fcj-noabs_lab6`, `tch-fcj_lab6`, - `tof_j_si`, `tof_jvd_si`, `sc-neut-cwl_ext-iso_tbti`. - - **`ci_skip.txt` only** (1): `tch-fcj-nosldl_lab6` (no flag → audit - as crash vs gated). - - **`raise_on_failure=False` only, not skip-listed** (3): - `pd-neut-tof_jvd_ncaf`, `sc-neut-cwl_noext_tbti`, - `sc-neut-cwl_pr2nio4` — these currently pass both runners ungated, - so the audit decides re-gate (`known_discrepancy=False`) vs keep - `known_discrepancy=True, reason=…`. -- **Migration is mechanical except the per-page verdict**, which - requires running each page once (disagree-cleanly vs crash; re-gate vs - keep). -- The non-asserting `verify.patterns_agree(...) -> bool` wrapper is - added **only if** a page or test needs a plain boolean (decide during - P1.1). - -## Open questions - -1. Per-page audit verdicts (P1.3) — **resolved**, recorded below. Each - page was run once with - `pixi run python docs/docs/verification/.py` against installed - cryspy 0.11.0 (the unpinned version CI also resolves), classifying by - exit code and the rendered agreement table: - - | Page | Result | Migration | - | --------------------------------------- | ------------------------------ | --------------------------------- | - | `pd-neut-cwl_pv-beba_pbso4` | agrees (fit recovers FullProf) | **re-gate** (drop flag) | - | `pd-neut-tof_jvd_ncaf` | agrees | **re-gate** (was not skip-listed) | - | `sc-neut-cwl_noext_tbti` | agrees | **re-gate** (was not skip-listed) | - | `sc-neut-cwl_pr2nio4` | agrees | **re-gate** (was not skip-listed) | - | `pd-neut-cwl_tch-fcj-noabs-nosldl_lab6` | disagrees, runs | `known_discrepancy=True` | - | `pd-neut-cwl_tch-fcj-noabs_lab6` | disagrees, runs | `known_discrepancy=True` | - | `pd-neut-cwl_tch-fcj_lab6` | disagrees, runs | `known_discrepancy=True` | - | `pd-neut-cwl_tch-fcj-nosldl_lab6` | disagrees, runs (note updated) | `known_discrepancy=True` | - | `pd-neut-tof_j_si` | crysfml disagrees | `known_discrepancy=True` | - | `pd-neut-tof_jvd_si` | cryspy disagrees | `known_discrepancy=True` | - | `sc-neut-cwl_ext-iso_tbti` | disagrees | `known_discrepancy=True` | - - No page crashed before its agreement check, so **no - `raises-exception` cell tags were needed**. The three not-skip-listed - pages (`jvd_ncaf`, `sc-noext`, `sc-pr2nio4`) all agree, so all are - re-gated. `pv-beba_pbso4` was previously skip-listed but now agrees - after its own-asymmetry fit, so it is re-gated too. - `tch-fcj-nosldl_lab6` previously asserted agreement - (`raise_on_failure=True`) but disagrees on released cryspy 0.11.0 - (its develop-cryspy demo agrees), so its note was updated and it is - marked `known_discrepancy=True`. - -2. `conftest.py`: **resolved at P1.2** — the file hosted only the - ci_skip xfail logic, so it was deleted entirely. - -## Concrete files likely to change - -- `src/easydiffraction/analysis/verification.py` — - `assert_patterns_agree` (flag, reason, two-sided assertion, return - contract). -- `tests/unit/easydiffraction/analysis/test_verification.py` — all - changes happen in **Phase 2**: migrate existing - `raise_on_failure`/return-contract cases and add the new - `known_discrepancy` cases. Phase 1 does not touch this file. -- `tools/test_scripts.py` — replace the `ci_skip.txt` skip with the - in-source static scan. -- `docs/docs/conftest.py` — remove the ci_skip xfail logic. -- `docs/docs/verification/ci_skip.txt` — **deleted**. -- The 11 audited `docs/docs/verification/*.py` pages (+ regenerated - `.ipynb` via `pixi run notebook-prepare`); also any other page - carrying a now-removable fit section (Decision 5). -- `docs/docs/verification/index.md` — wording that referenced CI-skip. -- `docs/dev/testing-guide.md` (~line 82) and the `verification-exec` - task comment in `pixi.toml` (~line 233) — `ci_skip` references that go - stale once the file is deleted; plus anything else the `git grep` - surfaces. -- `docs/dev/adrs/suggestions/verification-regression-flag.md` → `git mv` - to `accepted/`; `docs/dev/adrs/index.md` row flipped to Accepted; - links fixed. -- `docs/dev/adrs/accepted/test-suite-and-validation.md`, - `docs/dev/adrs/accepted/documentation-ci-build.md` — revise the - `ci_skip.txt` wiring text. -- `src/easydiffraction/analysis/analysis.py` line ~2819 — **Windows - fix** (see P1.7). - -Find every occurrence first with -`git grep -n "raise_on_failure\|ci_skip"`. - -## Implementation steps (Phase 1) - -Each `- [ ]` is one atomic commit; stage only the listed paths; commit -each before the next. - -- [x] **P1.1 — `known_discrepancy` in `assert_patterns_agree`.** In - `verification.py`: replace `raise_on_failure` with - `known_discrepancy: bool = False` and add - `reason: str | None = None` (required when - `known_discrepancy=True`); implement the two-sided assertion - (Decision 1a) and the "expectation met" return (Decision 1b); - raise a clear re-gate `AssertionError` on unexpected agreement. - Update the docstring only. **No `test_verification.py` changes in - Phase 1** — all test work (updating the existing - `raise_on_failure`/return-contract cases and adding the new - `known_discrepancy` cases) is the first task of Phase 2; the tests - may be left temporarily stale until then since they are not run - during Phase 1. Commit: - `Add known_discrepancy flag to assert_patterns_agree` - -- [x] **P1.2 — Remove `ci_skip.txt`; make both runners in-source.** - Delete `docs/docs/verification/ci_skip.txt`; remove the xfail - logic from `docs/docs/conftest.py` (delete the file if nothing - else lives there, per Open Question 2); replace the skip block in - `tools/test_scripts.py` with a static scan that skips a - verification `.py` declaring `known_discrepancy=True` or a - `raises-exception` tag. Also update the `ci_skip`-mechanism - references in `docs/dev/testing-guide.md` (~line 82) and the - `verification-exec` task comment in `pixi.toml` (~line 233), and - any other reference the grep surfaces, so no obsolete guidance is - left behind. Commit: - `Drive verification skips from in-source flag, drop ci_skip.txt` - -- [x] **P1.3 — Audit and migrate the 11 pages.** _(Named Phase-1 audit - exception — read-only classification, see intro.)_ Classify each - page by running its source once: - `pixi run python docs/docs/verification/.py` — a clean exit - means it runs to completion (then its agreement table says - disagree-cleanly vs already-agrees); an exception before the final - agreement call means a crash page. Record the per-page verdict in - this plan's Open Questions. Then for each: migrate - `raise_on_failure=False` → `known_discrepancy=True, reason=…` - (carrying the old `ci_skip.txt` reason onto the page) **or** - re-gate by dropping the flag where it actually agrees; add - `raises-exception` cell tags + notes for crash pages; remove fit - sections from pages that already agree (Decision 5). Run - `pixi run notebook-prepare`; stage the `.py` + regenerated - `.ipynb`. Commit: - `Migrate verification pages to known_discrepancy flag` - -- [x] **P1.4 — Update verification docs prose.** Fix - `docs/docs/verification/index.md` (and any page text) that - described the `ci_skip.txt` mechanism. Commit: - `Update verification docs for known_discrepancy gating` - -- [x] **P1.5 — Revise the two accepted ADRs.** Update - `test-suite-and-validation.md` §6 and `documentation-ci-build.md` - to the in-source mechanism (no `ci_skip.txt`). **Done:** §6 of - `test-suite-and-validation.md` gained a "Known-discrepancy gating" - bullet describing the in-source flag/tag and the two runners. A - `git grep` confirmed `documentation-ci-build.md` never referenced - `ci_skip.txt`/`conftest`, so it had no stale wiring to revise and - was left unchanged. Commit: - `Revise accepted ADRs for in-source verification gating` - -- [x] **P1.6 — Promote the ADR.** `git mv` - `docs/dev/adrs/suggestions/verification-regression-flag.md` → - `accepted/`; set `**Status:** Accepted`; flip its - `docs/dev/adrs/index.md` row to Accepted with the `accepted/...` - link; fix any links that pointed at `suggestions/...` - (`git grep -n verification-regression-flag`). Commit: - `Promote verification-regression-flag ADR to accepted` - -- [x] **P1.7 — Windows CI fix (bundled, unrelated).** In - `src/easydiffraction/analysis/analysis.py` (~line 2819) the - rewritten `data_dir` is built with - `str(Path('data') / 'sequential')`, which is `data\sequential` on - Windows and fails `test_copy_data_archives_and_rewrites_data_dir` - (expects `data/sequential`). Store a POSIX path: - `Path('data', 'sequential').as_posix()` (or the literal - `'data/sequential'`). The existing test is the spec — no test - change. Commit: - `Store sequential data_dir as POSIX path for Windows` - -- [x] **P1.8 — Phase 1 review gate.** No-code; mark `[x]` and commit the - checklist update alone. Commit: `Reach Phase 1 review gate` - -## Status checklist - -- [x] P1.1 known_discrepancy flag -- [x] P1.2 remove ci_skip.txt + runners -- [x] P1.3 audit + migrate pages -- [x] P1.4 verification docs prose -- [x] P1.5 revise accepted ADRs -- [x] P1.6 promote ADR -- [x] P1.7 Windows CI fix -- [x] P1.8 Phase 1 review gate -- [x] Phase 2 verification green - -## Phase 2 — Verification - -Per the `AGENTS.md` two-phase workflow, Phase 2 **starts with the test -work**, then runs the verification commands. No `test_verification.py` -edits happen in Phase 1 — Phase 1 may leave the existing tests -temporarily stale (they are not run until here). - -**Step 1 — update and extend `test_verification.py` first.** Before any -`pixi run` command: - -- **Update existing cases** to the changed contract: every test that - passed `raise_on_failure=` migrates to `known_discrepancy=`, and every - test asserting the old "returns the raw agreement boolean" return - value migrates to the Decision 1b "expectation met" contract (`True` - when the page met its expectation, `AssertionError` otherwise). -- **Add new cases** for the new behavior: - - the two-sided assertion — a `known_discrepancy=True` page that is - out of tolerance passes, and one that is within tolerance raises the - re-gate `AssertionError`; - - the required-`reason` validation — `known_discrepancy=True` without - a non-empty `reason` raises; - - the "expectation met" return value for both a default page and a - passing `known_discrepancy=True` page. - -**Step 2 — run the verification commands.** From the repo root; use the -zsh-safe log-capture when output is needed. - -- `pixi run fix` (commit auto-fixes incl. regenerated - `docs/dev/package-structure/{full,short}.md`). -- `pixi run check > /tmp/edi-check.log 2>&1; check_exit_code=$?; tail -n 200 /tmp/edi-check.log; exit $check_exit_code` -- `pixi run unit-tests > /tmp/edi-unit.log 2>&1; unit_tests_exit_code=$?; tail -n 120 /tmp/edi-unit.log; exit $unit_tests_exit_code` - (Linux cannot reproduce the Windows path failure — the - `test_copy_data_archives_and_rewrites_data_dir` test already passes - here; the P1.7 fix makes it pass on Windows too.) -- `pixi run integration-tests > /tmp/edi-int.log 2>&1; integration_tests_exit_code=$?; tail -n 120 /tmp/edi-int.log; exit $integration_tests_exit_code` -- `pixi run script-tests > /tmp/edi-script.log 2>&1; script_tests_exit_code=$?; tail -n 120 /tmp/edi-script.log; exit $script_tests_exit_code` - (must skip the `known_discrepancy=True` / `raises-exception` pages via - the new in-source scan — confirm none of them run here). -- `pixi run notebook-tests > /tmp/edi-nb.log 2>&1; notebook_tests_exit_code=$?; tail -n 120 /tmp/edi-nb.log; exit $notebook_tests_exit_code` - (every page executes; a still-discrepant `known_discrepancy=True` page - passes, and a `raises-exception` page xfails cell-precisely). - -## Suggested Pull Request - -**Title:** Simpler, self-documenting handling of known-bad verification -pages - -**Description:** A verification page that can't yet match its reference -(because an engine feature isn't implemented) is now marked **in the -notebook itself** with `known_discrepancy=True` and a short reason that -shows on the published page — replacing the separate hidden skip list. -Good pages stay regression tests as before; a known-bad page that later -starts matching now fails CI on purpose, so its "known-bad" mark gets -removed deliberately rather than lingering forever. Also fixes a -Windows-only path issue in sequential fitting so the saved data folder -is recorded consistently across operating systems. diff --git a/docs/dev/plans/verification-software-version-labels.md b/docs/dev/plans/verification-software-version-labels.md deleted file mode 100644 index 096bc7ffc..000000000 --- a/docs/dev/plans/verification-software-version-labels.md +++ /dev/null @@ -1,282 +0,0 @@ -# Plan: Software Version Labels on Verification Pages - -Follows [`AGENTS.md`](../../../AGENTS.md). No deliberate exceptions to -the two-phase workflow: Phase 1 is library + page edits only (no test -authoring), Phase 2 does all test work and runs the verification suite. - -## ADR - -Implements -[`verification-software-version-labels`](../adrs/accepted/verification-software-version-labels.md) -(Accepted). Per §Change Discipline it is **promoted to `accepted/`** as -part of this PR (P1.4). No new dependency is required — every version is -read through `easydiffraction.utils.utils.package_version` and formatted -for display by a small `verify` helper (`_label_version`, see -Decisions); `stripped_package_version` is **not** used, since it would -drop this repo's `+dev*`/`+dirty*`/`+devdirty*` markers. - -The sibling -[`verification-regression-flag`](../adrs/suggestions/verification-regression-flag.md) -ADR is **separate work**. Both touch the same 15 verification `.py` -pages and their regenerated notebooks, so see Open Question 1 on -ordering. - -## Branch + PR - -- Branch: `verification-software-version-labels` (flat slug off - `develop`). Created and checked out during the Phase 1 review-fix - cycle; the software-version-labels commits now live on this branch. -- PR targets `develop`, not `master`. Do not push unless asked. - -## Decisions (from the ADR) - -- **All three versions, both surfaces (Decision 1).** Each comparison - renders FullProf, EasyDiffraction, and the producing engine version in - **both** the plot legend (`reference_label` / `candidate_label`) - **and** the agreement table. The `assert_patterns_agree` row label is - a **combined** string `f'{candidate_label} vs {reference_label}'` so - all three appear there too (it takes one label per row, not a pair). -- **Bare `X.Y.Z` format (Decision 2).** No `v` prefix on any component. - `verify.fullprof_label` changes `FullProf vX.YZ` → `FullProf X.YZ`. - Candidate is `edi X.Y.Z (cryspy X.Y.Z)` / `edi X.Y.Z (crysfml X.Y.Z)`. -- **Dev/pre-release marker (Decision 2a).** Release plus short dev - marker, dropping the `+g` local segment: `edi 0.11.0.dev3`. -- **Helper takes an explicit engine tag (Decision 3).** - `verify.engine_label('cryspy', …)`, **not** - `experiment.calculator.type` read at render time, so a stored result - keeps the version of the engine that produced it. Candidate-only; the - reference side stays `verify.fullprof_label`. -- **Engine version source (Decision 3a).** Resolved through the existing - engine→package map + `importlib.metadata` path; a visible - `crysfml ?`-style marker when a version is unresolvable (never silent, - never a page hard-fail). -- **Refined / annotated candidates keep the version (Decision 4).** e.g. - `edi X.Y.Z (cryspy X.Y.Z, refined)`. - -## Decisions already made for this plan - -- **Dev-marker formatter keeps this repo's local markers; drops only a - git-hash tail.** This repo's versioningit config - (`pyproject.toml [tool.versioningit.format]`) carries the dev signal - in the **local** segment — `{base}+dev{N}`, `{base}+dirty{N}`, - `{base}+devdirty{N}` — which `stripped_package_version` would strip, - rendering `1.2.3+dev3` as `1.2.3` and hiding a dev build (caught in - review 1). So `engine_label` must **not** use - `stripped_package_version` for the EasyDiffraction version. Instead a - small `verify` formatter (e.g. `_label_version(pkg)`) takes the - **raw** `package_version` and **keeps** any - `+dev*`/`+dirty*`/`+devdirty*` marker, trimming **only** a pure - VCS-hash local part (`+g`) to honour Decision 2a's "drop the - noisy commit hash". Net effect on this repo: `1.2.3+dev3` → - `edi 1.2.3+dev3`; a clean release `1.2.3` → `edi 1.2.3`. The same - formatter is applied to engine versions. `_is_dev_version` in - `utils.utils` already encodes the `+dev`/`+dirty`/`+devdirty` marker - detection and can back the formatter. -- **Engine→package map moves to `utils.utils` (P1.1).** Today the map - lives as the private `_SOFTWARE_PACKAGE_BY_ENGINE` in `analysis.py` - (with `_software_version` reading the **raw** version for CIF - provenance stamping). To satisfy Decision 3a's "single resolution - path" without `verification.py` importing heavy `analysis.py`, the - **map** is extracted to `utils.utils` (which both modules already - import and which already owns `package_version`). `analysis.py` - imports it and keeps its existing raw-version stamping behavior - **unchanged**; `verification.py` reads the same map but formats the - result through the `_label_version` display helper (see the dev-marker - decision above). The format (raw for CIF vs display for labels) is a - per-caller choice on top of one shared map. -- **`refined: bool` is generalized to `note: str | None = None`.** Real - pages carry more than "refined": `(scale only)`, - `(scale + ext radius)`, `(refined)`. A boolean cannot express those, - so the helper signature is `engine_label(engine, note=None)` → - `edi X (cryspy X)` or `edi X (cryspy X, )`. `note='refined'` - covers Decision 4. This **refines** the ADR's `refined=False` wording; - the ADR's Decision 3/4 text is updated to `note=` during the promotion - step (P1.4) so ADR and code agree. -- **SC pages always get a versioned FullProf label — no bare fallback.** - Single-crystal pages currently pass the literal - `reference_label='FullProf'` and load - `verify.load_fullprof_sc_f2calc(dir, '.out')`. `fullprof_label` - accepts `.sum`/`.out`, and review 1 confirmed every SC `.out` in use - (`tbti.out`, `prnio.out`) carries the `FullProf.2k (Version 8.40 …)` - banner (sibling `.sum` files exist too). So all SC pages call - `verify.fullprof_label(FULLPROF_PROJECT_DIR, FULLPROF_OUT_FILE)` like - the powder pages. Keeping the bare `'FullProf'` is **not** an allowed - outcome (it would reintroduce the versionless state this work - removes); if a banner source is ever genuinely absent, the fallback is - a visible `FullProf ?`, never the bare label. - -## Open questions - -1. **Ordering vs `verification-regression-flag`.** Both edit all 15 - `docs/docs/verification/*.py` pages and regenerate the same - notebooks, and both are in flight. The Phase 1 review-fix cycle moved - this work onto its own `verification-software-version-labels` branch - from `develop`, so the PR can be reviewed independently. If the - regression-flag branch lands first, rebase this branch onto updated - `develop` and resolve the expected verification-page conflicts there. -2. **ADR Decision 4 wording.** The `refined`→`note` generalization above - edits the ADR at promotion (P1.4). Flagged so the owner can veto the - signature if a strict boolean is preferred. - -_(Resolved in review 1: the SC `.out` banner question — every -single-crystal `.out` in use carries the 8.40 banner, so SC pages use -`fullprof_label` with no versionless fallback; see the SC decision -above.)_ - -## Concrete files likely to change - -- `src/easydiffraction/utils/utils.py` — extract the engine→package map - (e.g. `SOFTWARE_PACKAGE_BY_ENGINE`) here; it already hosts - `package_version` / `stripped_package_version`. -- `src/easydiffraction/analysis/analysis.py` — import the shared map; - drop the local `_SOFTWARE_PACKAGE_BY_ENGINE`; `_software_version` - unchanged in behavior (still raw version for CIF stamping). -- `src/easydiffraction/analysis/verification.py` — add - `engine_label(engine, note=None)`; change `fullprof_label` to emit - `FullProf {version}` (drop `v`); update its docstring example. -- The **15** `docs/docs/verification/*.py` pages (+ regenerated `.ipynb` - via `pixi run notebook-prepare`): replace bare - `candidate_label='edi-cryspy'` / `'edi-crysfml'` / `'ed-cryspy'` / - `'ed-crysfml'` (and `(refined)` / `(scale only)` / - `(scale + ext radius)` variants) with `verify.engine_label(...)`; - switch literal `reference_label='FullProf'` to a page - `FULLPROF_LABEL = verify.fullprof_label(...)` where missing; build - combined `assert_patterns_agree` row labels. -- `tests/unit/easydiffraction/analysis/test_verification.py` — Phase 2 - only: update `test_fullprof_label_formats_version` (now - `FullProf 8.40`) and add `engine_label` cases. -- `docs/dev/adrs/suggestions/verification-software-version-labels.md` → - `git mv` to `accepted/`; `docs/dev/adrs/index.md` row flipped to - Accepted; links fixed. - -Find every label call site first with -`git grep -nE "candidate_label|reference_label|assert_patterns_agree|fullprof_label" -- 'docs/docs/verification/*.py'` -and every map user with `git grep -n "_SOFTWARE_PACKAGE_BY_ENGINE"`. - -## Implementation steps (Phase 1) - -Each `- [ ]` is one atomic commit; stage only the listed paths; commit -each before the next. No `test_verification.py` changes in Phase 1 — the -existing `fullprof_label` test may be left temporarily stale (it is not -run until Phase 2). - -- [x] **P1.1 — Share the engine→package map in `utils.utils`.** Move - `_SOFTWARE_PACKAGE_BY_ENGINE` from `analysis.py` into - `src/easydiffraction/utils/utils.py` (public - `SOFTWARE_PACKAGE_BY_ENGINE`); import it in `analysis.py` so - `_software_version` keeps its current raw-version behavior - unchanged. Pure refactor, no behavior change. Commit: - `Share engine-to-package map from utils` - -- [x] **P1.2 — Add `engine_label`; make `fullprof_label` bare.** In - `verification.py`: add a `_label_version(pkg)` formatter (raw - `package_version`, keep `+dev*`/`+dirty*`/`+devdirty*`, trim only - a `+g` VCS-hash tail — see the dev-marker decision above) and - `engine_label(engine, note=None)` that builds - `edi {edi_ver} ({engine} {engine_ver}[, {note}])` via that - formatter and `SOFTWARE_PACKAGE_BY_ENGINE`, rendering a visible - `{engine} ?` marker when the engine version is `None` (Decision - 3a). Change `fullprof_label` to return `f'FullProf {version}'` and - update its docstring example. Docstrings only; no test edits. - Commit: `Add engine_label helper and drop v from fullprof_label` - -- [x] **P1.3 — Migrate the 15 verification pages.** For each - `docs/docs/verification/*.py`: define page-level label variables - right after each calculation - (`LABEL_CRYSPY = verify.engine_label('cryspy')`, refined/annotated - via `note=`), set - `FULLPROF_LABEL = verify.fullprof_label(, )` - where the reference label is still the literal `'FullProf'`, pass - these to `pattern_comparison` / `reflection_comparison`, and build - each `assert_patterns_agree` row label as - `f'{LABEL_xxx} vs {FULLPROF_LABEL}'`. SC pages use - `fullprof_label` with their `.out` (banners confirmed in review 1) - — no bare `'FullProf'` left anywhere. Run - `pixi run notebook-prepare`; stage the `.py` + regenerated - `.ipynb`. Commit: `Show software versions on verification pages` - -- [x] **P1.4 — Promote the ADR.** First reconcile the ADR text with what - was implemented: (a) update Decision 3/4 wording from - `refined=False` to `note=None` (Open Question 3); (b) update - **Decision 3a** and its matching resolved Open Question so they - name the new shared map location (`SOFTWARE_PACKAGE_BY_ENGINE` in - `utils.utils`, P1.1) instead of the old private `analysis.py` - `_SOFTWARE_PACKAGE_BY_ENGINE`/`_software_version`, and record the - per-caller raw-vs-display formatting split (analysis stamps the - raw version into CIF; `verify` renders the dev-marker display form - via `_label_version`); (c) revise **Decision 2a** and its matching - resolved format Open Question so they describe the implemented - display formatter — preserve this repo's configured - `+dev*`/`+dirty*`/ `+devdirty*` local markers and trim **only** a - pure `+g` VCS-hash suffix (the `.devN+g…` example may stay as - the hash-trim illustration). Then `git mv` - `docs/dev/adrs/suggestions/verification-software-version-labels.md` - → `accepted/`; set `**Status:** Accepted`; flip its - `docs/dev/adrs/index.md` row to Accepted with the `accepted/...` - link; fix any links that pointed at `suggestions/...` - (`git grep -n verification-software-version-labels`). Commit: - `Promote verification-software-version-labels ADR to accepted` - -- [x] **P1.5 — Phase 1 review gate.** No-code; mark `[x]` and commit the - checklist update alone. Commit: `Reach Phase 1 review gate` - -## Status checklist - -- [x] P1.1 share engine→package map -- [x] P1.2 engine_label + bare fullprof_label -- [x] P1.3 migrate 15 verification pages -- [x] P1.4 promote ADR -- [x] P1.5 Phase 1 review gate -- [ ] Phase 2 verification green - -## Phase 2 — Verification - -Per the two-phase workflow, Phase 2 **starts with the test work**, then -runs the verification commands. - -**Step 1 — update and extend `test_verification.py` first.** Before any -`pixi run` command: - -- **Update existing case.** `test_fullprof_label_formats_version` must - now expect `'FullProf 8.40'` (no `v`). -- **Add `engine_label` cases:** - - basic candidate string for a known engine (assert the - `edi X.Y.Z (cryspy X.Y.Z)` shape using the installed versions); - - `note='refined'` appends `, refined`; - - an unresolvable engine version renders the visible `?` marker, not a - silent omission and not a raise (monkeypatch the version lookup to - return `None`); - - the dev-marker behavior for this repo's configured forms - (monkeypatch the lookup): `1.2.3+dev3`, `0.5.8+dirty3`, and - `0.5.8+devdirty3` are **preserved** verbatim in the label (a dev - build never collapses to its release number), while a pure VCS-hash - tail such as `1.2.3+g1a2b3c` is trimmed to `1.2.3`. - -**Step 2 — run the verification commands.** From the repo root; use the -zsh-safe log-capture when output is needed. - -- `pixi run fix` (commit auto-fixes incl. regenerated - `docs/dev/package-structure/{full,short}.md`). -- `pixi run check > /tmp/edi-check.log 2>&1; check_exit_code=$?; tail -n 200 /tmp/edi-check.log; exit $check_exit_code` -- `pixi run unit-tests > /tmp/edi-unit.log 2>&1; unit_tests_exit_code=$?; tail -n 120 /tmp/edi-unit.log; exit $unit_tests_exit_code` -- `pixi run integration-tests > /tmp/edi-int.log 2>&1; integration_tests_exit_code=$?; tail -n 120 /tmp/edi-int.log; exit $integration_tests_exit_code` -- `pixi run script-tests > /tmp/edi-script.log 2>&1; script_tests_exit_code=$?; tail -n 120 /tmp/edi-script.log; exit $script_tests_exit_code` - (every verification page now reads live versions; confirm none fail on - a missing FullProf banner — Open Question 2). -- `pixi run test-structure-check` (confirms no new test-layout drift - from the added `engine_label` cases). - -## Suggested Pull Request - -**Title:** Every verification page now shows the exact software versions -behind its curves - -**Description:** Each cross-engine verification page now states which -software produced the compared curves — the FullProf version, the -EasyDiffraction version, and the calculation engine (cryspy or crysfml) -version — right in the plot legend and the agreement table (for example -`edi 0.11.0 (cryspy 2.4.1) vs FullProf 8.40`). Because cross-engine -agreement can depend on the exact build, this makes every comparison -reproducible and lets a reader tell at a glance what was compared, even -after a later software update. The versions are read live, so they stay -correct automatically whenever a component is upgraded. diff --git a/docs/dev/plans/xray-cw-polarization-optics.md b/docs/dev/plans/xray-cw-polarization-optics.md deleted file mode 100644 index cb4148cc9..000000000 --- a/docs/dev/plans/xray-cw-polarization-optics.md +++ /dev/null @@ -1,288 +0,0 @@ -# Implementation Plan: X-ray CW Polarization Optics - -This plan follows the conventions in [`AGENTS.md`](../../../AGENTS.md). -No deliberate exceptions to those instructions are taken. Per the -two-phase workflow, **Phase 1 is code + docs only** (no test creation -unless explicitly requested); tests and the five `pixi` verification -tasks run in **Phase 2**. When an AI agent executes this plan, **every -completed Phase 1 step must be staged with explicit paths and committed -locally before the next step** (atomic, single-purpose commits per -§Commits), and the agent stops at the Phase 1 review gate. - -## ADR - -Implements -[`docs/dev/adrs/accepted/xray-cw-polarization-optics.md`](../adrs/accepted/xray-cw-polarization-optics.md) -(Status: Accepted). The ADR was promoted to `accepted/` during P1.1 per -§Change Discipline. This plan uses the same slug as the ADR. - -Related accepted ADRs consulted: - -- [`model-sample-absorption.md`](../adrs/accepted/model-sample-absorption.md) - — the precedent for a CW-powder physics correction applied as a shared - pointwise multiplier after the engine returns (`absorption.apply`). -- [`immutable-experiment-type.md`](../adrs/accepted/immutable-experiment-type.md) - — `radiation_probe` is a creation-time axis, persisted in - `_expt_type.radiation_probe` and restored before the instrument is - rebuilt. -- [`iucr-cif-tag-alignment.md`](../adrs/accepted/iucr-cif-tag-alignment.md) - — experiment-tier fields keep short `_instr.*` names in the default - save. - -## Branch and PR - -- Flat-slug implementation branch off `develop`: - `xray-cw-polarization-optics` (created/checked out by - `/draft-impl-1`). -- PR targets `develop`, not `master`. Do not push unless asked. - -## Dependencies - -No new runtime or dev dependencies. Cryspy already declares `k`/`cthm` -on its `Setup` item (defaults `k=0.0`, `cthm=0.91`), so binding is value -injection, not a backend extension. No `pyproject.toml` / `pixi.toml` / -`pixi.lock` changes are expected. - -## Decisions (already made in the ADR) - -1. The two values live on `experiment.instrument` (not a new optics - category, not a peak-shape model). -2. The CW **powder Bragg** instrument splits by `radiation_probe`: - `CwlPdInstrument` (tag `cwl-pd`) becomes `CwlPdNeutronInstrument` - (`cwl-pd-neutron`) and `CwlPdXrayInstrument` (`cwl-pd-xray`). The - X-ray class owns the optics fields; the neutron class does not. - Single crystal (`cwl-sc`), TOF (`tof-pd`/`tof-sc`), and - total-scattering PDF are unchanged. The identically named - **data-range** `cwl-pd` tag is out of scope and stays. Both new - classes carry the full **Bragg-only** compatibility tuple — - `sample_form={POWDER}`, `scattering_type={BRAGG}`, - `beam_mode={CONSTANT_WAVELENGTH}`, plus their `radiation_probe` — and - `calculator_support={CRYSPY, CRYSFML}`. The old class's `TOTAL` / - `PDFFIT` metadata is dropped because total-scattering PDF never - routes through this instrument (the ADR calls those entries unused - aspirational metadata); empty `Compatibility` axes mean "any", so the - tuple must be stated in full to avoid silently matching every sample - form / beam mode. -3. Public names are physical and non-refinable (`NumericDescriptor`): - - `setup_polarization_coefficient` — dimensionless, default `0.0`, - range `[0, 1]` → cryspy `_setup_K` / FullProf `Rpolarz`. - - `setup_monochromator_twotheta` — degrees, default `0.0`, range - `[0, 180)` → backend `cthm = cos²(radians(2θm))` (default `0.0°` ⇒ - `cthm = 1.0`, "no monochromator"). -4. **cryspy binds natively**: emit `_setup_K` / `_setup_cthm` in the - generated CIF (and patch the cached dict like other instrument - scalars). cryspy applies the Lorentz-polarization factor internally; - the EasyDiffraction multiplier is **not** applied to cryspy output. -5. **crysfml binds via a post-pattern multiplier**, but only _after_ the - ADR Decision 5 gate is satisfied: the implementer first verifies - (static source/CFL-grammar inspection) whether a native - polarization/monochromator CFL line exists. If one does, that native - line is emitted (and the ADR/plan updated); otherwise — the expected - outcome from the bundled CFL examples — - `y *= lp_factor(two_theta, k, cthm)` is applied after the engine - returns, in a shared, unit-tested helper. Two shared pieces: - `monochromator_cthm(angle)` (used by both adapters to produce the - backend `cthm`) and `lp_factor(two_theta, k, cthm)` (crysfml runtime - multiplier + the oracle that verifies cryspy's native output). P1.5 - inspection found lower-level CrysFML Lorentz routines with optional - `cthm`/`rkk` arguments, but the Python `cw_powder_pattern_from_dict` - wrapper reads no dictionary keys for them, so the planned fallback - multiplier is used. -6. Defaults are neutron-neutral: with the coefficient at `0.0`, `hh = 1` - and results are unchanged until a user opts in. - -## Open questions - -1. **Verification reference data (P1.6).** Resolved in P1.6: no new - FullProf polarized reference data was present, so the implementation - used the plan's lighter demonstration path in - `docs/docs/tutorials/simulate-nacl-xray.py` and regenerated the - matching notebook. -2. **cryspy cached-dict keys (P1.4).** Resolved in P1.4: the CIF-build - path emits `_setup_K` and `_setup_cthm`, the cached working dict is - patched when `k`/`cthm` keys are exposed, and polarization settings - are part of the cache-invalidation key so releases without those - patch keys rebuild from CIF on value changes. - -## Concrete files likely to change - -Phase 1 (code + docs): - -- `src/easydiffraction/datablocks/experiment/categories/instrument/cwl.py` - — split `CwlPdInstrument` into `CwlPdInstrumentBase` + - `CwlPdNeutronInstrument` + `CwlPdXrayInstrument`; declare the two - descriptors and their properties on the X-ray class. -- `.../instrument/factory.py` — replace the single powder-CW rule with - two `radiation_probe`-keyed rules. -- `.../instrument/__init__.py` — import/register the two new concrete - classes (drop the old `CwlPdInstrument` import). -- `src/easydiffraction/datablocks/experiment/item/bragg_pd.py` — pass - `radiation_probe=self.experiment_type.radiation_probe.value` into - `InstrumentFactory.default_tag(...)`. -- `src/easydiffraction/analysis/calculators/polarization.py` — **new** - shared helper: `monochromator_cthm`, `lp_factor`, `apply`. -- `src/easydiffraction/analysis/calculators/cryspy.py` — emit `_setup_K` - / `_setup_cthm` in `_cif_instrument_section`; optional cached-dict - patch in `_update_experiment_in_cryspy_dict`. -- `src/easydiffraction/analysis/calculators/crysfml.py` — apply - `polarization.apply(y, experiment)` on the return path (~line 170). -- A tutorial/verification source `.py` (P1.6, pending open question 1) - plus its regenerated notebook via `pixi run notebook-prepare`. -- ADR move to `accepted/` + `docs/dev/adrs/index.md` row. - -Phase 2 (tests + verification): - -- `tests/unit/.../instrument/test_factory.py`, - `tests/unit/.../analysis/calculators/test_support.py`, - `tests/unit/.../instrument/test_cwl.py`, and a new - `tests/unit/.../analysis/calculators/test_polarization.py`. -- `docs/dev/package-structure/{full,short}.md` (regenerated by - `pixi run fix`). - -## Implementation steps (Phase 1) - -- [x] **P1.1 — Promote the ADR to `accepted/`.** The ADR was moved into - `docs/dev/adrs/accepted/`, its status was set to `Accepted`, the - existing **Experiment model** row in `docs/dev/adrs/index.md` was - repointed to `accepted/xray-cw-polarization-optics.md`, this - plan's `## ADR` link was updated, and remaining stale path - references were cleared. Commit: - `Promote xray-cw-polarization-optics ADR to accepted` - -- [x] **P1.2 — Split the CW powder instrument by radiation probe.** In - `cwl.py`, extract a `CwlPdInstrumentBase(CwlInstrumentBase)` - holding the existing `calib_*` fields and properties. Add - `CwlPdNeutronInstrument` (tag `cwl-pd-neutron`) and - `CwlPdXrayInstrument` (tag `cwl-pd-xray`). Give **both** the full - Bragg-only `Compatibility`: - `sample_form=frozenset({SampleFormEnum.POWDER})`, - `scattering_type=frozenset({ScatteringTypeEnum.BRAGG})`, - `beam_mode=frozenset({BeamModeEnum.CONSTANT_WAVELENGTH})`, and - `radiation_probe=frozenset({RadiationProbeEnum.NEUTRON})` / - `{RadiationProbeEnum.XRAY})` respectively; and - `calculator_support=CalculatorSupport(calculators=frozenset( {CalculatorEnum.CRYSPY, CalculatorEnum.CRYSFML}))` - — **no `PDFFIT`, no `TOTAL`**. On the X-ray class declare the two - `NumericDescriptor`s (with `DisplayHandler`, `AttributeSpec` - defaults/validators, and `TagSpec` exactly as ADR Decision 3) plus - their getter/setter properties (mirror the `setup_wavelength_2` - property style). In `factory.py` replace the `cwl-pd` rule with - two rules keyed on `radiation_probe`. In `bragg_pd.py` pass - `radiation_probe=...` to `default_tag(...)`. Update - `instrument/__init__.py` registration. Leave the data-range - `cwl-pd` tag untouched. Commit: - `Split CW powder instrument by radiation probe` - -- [x] **P1.3 — Add the shared Lorentz-polarization helper.** New module - `analysis/calculators/polarization.py` mirroring `absorption.py`: - `monochromator_cthm(twotheta_deg) -> float` - (`cos²(radians(twotheta))`), - `lp_factor(two_theta, k, cthm) -> np.ndarray` - (`1 - k + k*cthm*cos²(two_theta)`), and - `apply(y, experiment) -> np.ndarray` that no-ops unless the - instrument exposes `setup_polarization_coefficient` with a - non-zero value, otherwise multiplies `y` by `lp_factor` over the - experiment's 2θ grid. Commit: - `Add shared Lorentz-polarization helper` - -- [x] **P1.4 — Bind cryspy natively.** In `cryspy.py` - `_cif_instrument_section`, within the existing powder branch, when - `hasattr(instrument, 'setup_polarization_coefficient')` append - `_setup_K ` and - `_setup_cthm ` - (import `monochromator_cthm` from P1.3). Do **not** apply - `polarization.apply` to cryspy output. If the cached working dict - in `_update_experiment_in_cryspy_dict` exposes the keys, patch - them with a presence guard (mirroring `offset_sycos`); otherwise - document reliance on the CIF-build path + cache rebuild (open - question 2). Commit: - `Emit cryspy polarization setup for X-ray CW powder` - -- [x] **P1.5 — Bind crysfml (verify native line first, then fall - back).** Satisfy the ADR Decision 5 gate explicitly: statically - inspect the active CrysFML CFL grammar / bundled examples for a - native polarization or monochromator line. **If one exists**, emit - that native line and update the ADR/plan to record it (or stop and - ask if it reopens scope). **If none exists** (the expected - outcome), apply `polarization.apply(y, experiment)` on the crysfml - return path alongside the existing - `absorption_correction.apply(...)` (~`crysfml.py:170`); no-op for - neutron / zero coefficient. State the inspection result in the - commit body. Commit: - `Apply Lorentz-polarization envelope on crysfml CW powder` - -- [x] **P1.6 — Demonstrate the new fields in docs.** Per open question - 1: either add the `pd-xray-pbso4-single-polarized-wdt48.py` - verification source (if the FullProf reference data is available) - or set `setup_polarization_coefficient` / - `setup_monochromator_twotheta` in an existing X-ray CW source and - show the pattern responds. Edit the `.py` source only, then run - `pixi run notebook-prepare` and commit the source plus regenerated - notebook. Commit: `Document X-ray CW polarization optics fields` - -- [x] **P1.7 — Phase 1 review gate.** No-code step. Mark complete and - commit the checklist update alone. Commit: - `Reach Phase 1 review gate` - -## Phase 2 — Verification - -Add/update tests, then run the five tasks. Capture logs with the -zsh-safe pattern where output is needed for analysis. - -Tests to add/update: - -- `test_factory.py` — `default_tag(..., radiation_probe=NEUTRON|XRAY)` - resolves to `cwl-pd-neutron` / `cwl-pd-xray`; `create()` for both - tags; remove `'cwl-pd'` instrument assertions. -- `test_support.py` — replace the `cwl-pd` calculator-support assertion - with the two new tags, asserting **only** the Bragg engines - (`{CRYSPY, CRYSFML}`) for `cwl-pd-neutron` / `cwl-pd-xray` (no - `PDFFIT`). -- `test_cwl.py` — the X-ray class exposes both descriptors with correct - defaults (`0.0`, `0.0`), range validation rejects out-of-range/wrong - type (`typeguard.TypeCheckError`), and the neutron class does **not** - expose them; CIF round-trip of `_instr.polarization_coefficient` / - `_instr.monochromator_twotheta`. -- new `test_polarization.py` — `monochromator_cthm(0) == 1.0`, the - `cos²` identity at a known angle, `lp_factor` reduces to `1` when - `k == 0`, and `apply` is a no-op for the neutron instrument. -- Do **not** modify the data-range `test_factory.py` `cwl-pd` cases. - -Commands: - -```bash -pixi run fix -pixi run check > /tmp/edi-check.log 2>&1; check_exit_code=$?; tail -n 200 /tmp/edi-check.log; exit $check_exit_code -pixi run unit-tests > /tmp/edi-unit.log 2>&1; unit_tests_exit_code=$?; tail -n 200 /tmp/edi-unit.log; exit $unit_tests_exit_code -pixi run integration-tests > /tmp/edi-integration.log 2>&1; integration_tests_exit_code=$?; tail -n 200 /tmp/edi-integration.log; exit $integration_tests_exit_code -pixi run script-tests > /tmp/edi-script.log 2>&1; script_tests_exit_code=$?; tail -n 200 /tmp/edi-script.log; exit $script_tests_exit_code -``` - -`pixi run fix` regenerates `docs/dev/package-structure/{full,short}.md`; -include them only in the `pixi run fix` commit. Leave generated -`docs/dev/benchmarking/*.csv` untracked unless asked. - -## Status checklist - -- [x] P1.1 — Promote the ADR to `accepted/` and update `index.md` -- [x] P1.2 — Split the CW powder instrument by radiation probe -- [x] P1.3 — Add the shared Lorentz-polarization helper -- [x] P1.4 — Bind cryspy natively -- [x] P1.5 — Bind crysfml (verify native line first, then fall back) -- [x] P1.6 — Demonstrate the new fields in docs -- [x] P1.7 — Phase 1 review gate -- [x] Phase 2 — tests added/updated and all five tasks pass - -## Suggested Pull Request - -**Title:** Match FullProf X-ray intensities with polarization and -monochromator settings - -**Description:** Constant-wavelength X-ray powder experiments now expose -two incident-optics controls — a polarization coefficient and a -pre-sample monochromator angle — so calculated intensities can match the -Lorentz-polarization correction used by FullProf and other Rietveld -codes. The settings appear only on X-ray powder instruments and are -applied by both the CrysPy and CrysFML engines; neutron experiments are -unaffected, and existing X-ray results stay the same until you set a -polarization value. This closes a known gap behind the residual -intensity mismatch seen in the PbSO₄ X-ray verification.