Rename period to era and pass a Corrections instance to the analysis hooks

[kandrosov]

Summary

The FLAF part of the multi-era change set. It does two things: the period → era rename, and an explicit Corrections instance instead of a global one. Single-era behaviour and every output path and file name are unchanged. The core bundle hash changes.

For users:

• --era Run3_2022EE replaces --period Run3_2022EE. These forms stop with a message naming the replacement:
  • --period, --<Task>-period and [<Task>] period in a law config: 'period' was renamed to 'era' (--era);
  • a period= keyword: 'period' was renamed to 'era' (use era=...).
• The era is set once per invocation: --<Task>-era and [<Task>] era are refused too. Several eras (--era A,B) are refused for now (several eras in one task are not supported yet); this is what the next step of the change set enables. An unknown era lists the known ones.
• Bundle patterns use {era}; {period} is refused.
• The payload scripts take --era.
• law index has to be re-run once (task ids change).
• Updating a checkout: stop the law drivers first, and let non-bundle HTCondor jobs of the old code finish. Such a job runs the checkout's code and was submitted with --period, which the new code refuses.

For analysis code (companion PRs): the hooks get the Corrections instance as a required keyword:

• addAllVariables(..., *, corrections);
• GetDfw(df, setup, dataset_name, *, corrections[, stage]);
• DefineWeightForHistograms(..., corrections).

Utilities.CreateCorrections(setup, dataset_name, stage, aggregated_caches=None) builds the instance for the HistTuple and AnalysisCache payloads and keeps it on the DataFrameWrapper (dfw.corrections) for as long as the dataframe lives. anaTupleProducer and MergeAnaTuples construct theirs directly. The AnaTuple column named period keeps its name.

Inputs: the existing AnaTuples are found and read by path. The v2610 AnaTuples as input are checked below.

Changes

• run_tools/law_customizations.py:
  • EraParameter: a sorted, unique CSV parameter, limited to one era for now;
  • a period tombstone and the command-line/config scan;
  • setup as a property and setup_for(era);
  • {era} in bundle patterns; the era string in every path, never the tuple.
• Common/Setup.py: Setup(era). Common/Utilities.py: CreateCorrections, dfw.corrections.
• AnaProd/, Analysis/, Common/: self.era[0] and --era everywhere; the hooks get corrections=.
• run_tools/check_era_rename.py, with test/test_check_era_rename.py: the leftover check (period identifiers outside an allow-list, --period, {period}, the era tuple used as a string). It runs in unit-tests.yaml over FLAF's own files, and the other repos will run it from FLAF main once this PR is merged.
• Tests:
  • test/test_era_parameter.py: identity, round trip, every refused form including {period} in a bundle pattern;
  • the existing tests are ported to era=.
• Docs (they deploy from main when this merges): the era pages, the arguments, troubleshooting, the architecture (Corrections in the hooks), the integration pipeline (the GitLab test_corrections job), GitHub Actions (the rename check); .github/copilot-instructions.md.

Companions: cms-flaf/Corrections#144 (merge first), cms-flaf/flaf_integration!33, cms-flaf/StatInference#27, cms-flaf/HH_bbtautau#98, cms-flaf/HH_bbWW#167, cms-flaf/H_mumu#80.

Testing

• Unit tests (the list in unit-tests.yaml), at 3e930ed: 762 passed, 807 subtests, 0 skipped in flaf_env on lxplus. At a67b5c9, before main (Ship the existing outputs of the submitted branches' requirements in the CRAB path-cache snapshot #346) was merged in, 753 passed both there and in a ROOT-free venv built like the workflow's. The rename check over FLAF's own files gives rc 0, and the docs build passes with mkdocs build --strict.
• main was merged in at 64323ff for Ship the existing outputs of the submitted branches' requirements in the CRAB path-cache snapshot #346 (the CRAB path-cache snapshot). Its new test task now uses self.era[0] (00acacb). Ship the existing outputs of the submitted branches' requirements in the CRAB path-cache snapshot #346's code runs here with the era parameter in its own unit tests (TheJobChecksItsRequirementsFromTheSnapshot). The HTCondor and CRAB chains below ran before Ship the existing outputs of the submitted branches' requirements in the CRAB path-cache snapshot #346 existed, at a67b5c9 and 1021607.
• Old vs new identity, 3 analyses × 7 eras: two trees pinned to commits. "old" is FLAF ff9fb55, Corrections 8ffb871 and the analyses' main; "new" is this change set before review round 1, which changes only formatting, docs, tests and one column name in bTagEffprod.py. run_ci_test.sh <ana> <era> htcondor (TestModel, --test 1000), then every ROOT file compared column by column and histogram by histogram, and the JSONs without timing fields: all 42 chains completed with RC 0.
  • H_mumu and HH_bbWW: identical in 7 of 7 eras (every column, every histogram, every JSON apart from timing fields).
  • HH_bbtautau: Run3_2022, 2024 and 2025 are identical. In Run3_2022EE, 2023, 2023BPix and 2026, only the AnaTuples' HHbtag scores differ (b1_HHbtag, b2_HHbtag, VBFJet_HHbtag and their shifts, |diff| <= 2.4e-4); HistTuples and histograms are identical.
  • Those HH_bbtautau pairs ran on worker nodes of different CPU families, and the identical pairs on the same family. Old vs new re-run on one node is identical, and the old code run twice on two CPU types differs in the same way. The differences come from CPU-dependent inference, not from this change.
  • HH_bbWW with the production model (Run3_Model, Run3_2022EE; a signal, TT, DY and a data stream): identical.
• v2610 AnaTuples as input (HH_bbWW, Run3_2022–2023BPix): HistTuple and AnalysisCache (DNN, DeepHME, BtagShape) on the production v2610 AnaTuples with --ana-version v2610, old vs new. Both find the v2610 merge outputs complete at the same URIs. Result: Run3_2023 is identical. In Run3_2022, 2022EE and 2023BPix only the DeepHME cache columns differ (about 1e-5 relative; the integer mass is off by 1 in a few events), and the HistTuples are identical. The old code run twice on two CPU types differs in the same way, and old vs new re-run on one node is identical. Both versions found every v2610 AnaTuple-side task complete, at identical merge-output URIs. AnaTupleFileListTask, which keeps a local copy in data/, was run first, on both sides.
• Layout (branch maps, branch data including out_name, every output URI and its existence, store/log/heartbeat/worker-file directories, bundle URIs, job-file names, CRAB request-name prefix), old vs new: identical for 415 stage lines, apart from the core hash.
  • Coverage: TestModel, 3 analyses × 7 eras; HH_bbWW v2610 (Run3_2022, 2022EE, 2023: AnaTupleFile, Builder, FileList, Merge; 2023BPix FileList/Merge); HH_bbtautau v2610 Run3_2024 AnaTupleFileTask and cost probes. The Run3_2024 input lists were regenerated from DAS; 135 of 137 datasets match production, the other two are DAS datasets that are now invalid.
  • Negative controls (era string, dataset order) make the diff non-empty.
• Two eras in one process: createAnatuple (H_mumu, HH_bbWW, HH_bbtautau) and mergeAnaTuples give the same output as separate processes. The old code leaks the first era's tau ES, PU, b-tag and JEC into the second. With the instance kept only on dfw.corrections, values are identical to keeping a local reference.
• run_ci_test.sh <ana> Run3_2022EE htcondor at the final commits (FLAF a67b5c9, Corrections be08572):
  • HH_bbtautau accea42 (afinal_HH_bbtautau_Run3_2022EE_10180010), HH_bbWW f0c8eb0 (afinal_HH_bbWW_Run3_2022EE_10180010, plus Run3_2022) and H_mumu 24c893a (afinal_H_mumu_Run3_2022EE_10180010) all completed with RC 0 and no failed task;
  • plots under /eos/user/k/kandroso/FLAF_all/<ana>/<version>/Plots/.
• CRAB, Run3_2022EE, on the tree from before review round 1 (FLAF 1021607, Corrections a9ca7e4; round 1 changed only formatting, docs, tests and the bTagEffprod column name). Runs: run_ci_test.sh <ana> Run3_2022EE crab for the three analyses (aval_crab_<ana>_10172044), and with test_crab_prodfs (fs_default T3_US_FNALLPC:/store/user/lpcflaf/FLAF_ci_prodfs/<ana>/) for HH_bbtautau and HH_bbWW (aval_prodfs_<ana>_10172044).
  • All five completed with RC 0. Every CRAB task reached SUBMITTED on the scheduler and its jobs finished.
  • 8 jobs failed on the worker side with exit code 5, at T2_CH_CSCS (bundle download: Host is down), T2_US_Florida (a node with a broken interpreter), T3_UK_London_RHUL, T3_KR_KNU and T2_US_Caltech. law resubmitted each one, and the retries succeeded.
• HelloWorld and the refused --period on the real CLI: HelloWorld with --era passes for the three analyses: local success, local force-fail and HTCondor success. The HTCondor force-fail and --bundle variants fail the same way on the old code, so they are pre-existing and not caused by this change. The test's log regex keeps the trailing comma of law's log: <uri>, remote host: … line, and the bundle variant has failed on main since at least f642ddb. law run FLAF.AnaProd.tasks.AnaTupleFileTask --period Run3_2022EE --print-status 0 stops with ValueError: --period: 'period' was renamed to 'era' (--era), and --era works.
• Gate: REVIEWER OK (flaf-reviewer, a67b5c9); TESTER VERDICT: PASS (htcondor, HH_bbtautau / HH_bbWW / H_mumu, FLAF a67b5c9 / Corrections be08572); TESTER VERDICT: PASS (crab and test_crab_prodfs, FLAF 1021607 / Corrections a9ca7e4).
• Integration pipelines with every companion: triggered on Rename period to era and pass a Corrections instance to the analysis hooks #347 with every companion (FLAF 3e930ed, Corrections be08572, StatInference e1392d7, HH_bbtautau accea42, HH_bbWW f0c8eb0, H_mumu 24c893a, 7 eras).
  • GitLab, with flaf_integration!33: parent https://gitlab.cern.ch/cms-flaf/flaf_integration/-/pipelines/16078958, child https://gitlab.cern.ch/cms-flaf/flaf_integration/-/pipelines/16078964. 117 of 117 jobs successful, including test_corrections (47 of 47 at Corrections be08572); every verify_revision names the heads above.
  • GitHub Actions: https://github.com/cms-flaf/FLAF/actions/runs/39089883382, 118 of 118 jobs successful.
  • Every test job runs --era <its era>; no job uses --period.

[kandrosov]

@cms-flaf-bot please test

• Own every correction provider per Corrections instance; rename period to campaign and era Corrections#144
• Rename period to era StatInference#27
• Rename period to era and take the Corrections instance in every hook HH_bbtautau#98
• Rename period to era and take the Corrections instance in every hook HH_bbWW#167
• Rename period to era and take the Corrections instance in every hook H_mumu#80
• gitlab_branch=era-rename

[kandrosov]

@cms-flaf-bot please test

• Own every correction provider per Corrections instance; rename period to campaign and era Corrections#144
• Rename period to era StatInference#27
• Rename period to era and take the Corrections instance in every hook HH_bbtautau#98
• Rename period to era and take the Corrections instance in every hook HH_bbWW#167
• Rename period to era and take the Corrections instance in every hook H_mumu#80
• ci_backend=github

[cms-flaf-bot]

pipeline#16078958 started

[cms-flaf-bot]

GitHub Actions integration workflow dispatched for HH_bbWW, HH_bbtautau, H_mumu

[cms-flaf-bot]

pipeline#16078958 passed

• HH_bbWW (PR #167): f0c8eb0
• HH_bbtautau (PR #98): accea42
• H_mumu (PR #80): 24c893a
• FLAF (PR #347): 3e930ed
• Corrections (PR #144): be08572
• StatInference (PR #27): e1392d7

[cms-flaf-bot]

✅ GitHub Actions integration pipeline passed.