v492.0 dev7

[GernotMaier]

This pull request introduces several improvements and updates to the Monte Carlo (MC) vs. data comparison workflow, environment setup, and documentation for the release tests. The main changes include enhanced automation and robustness in the MC/data comparison scripts, improved documentation for users, updates to environment and pre-commit configurations, and minor bug fixes and clarifications.

MC/Data Comparison Workflow Improvements:

• The compareDatawithMC.sh script now robustly handles parameter parsing with awk, supports both atmosphere-specific and generic run lists, manages multiple output directories, and enforces the presence of required files (including new support for XGB stereo files when using the XGB method). It also adds improved error handling and more flexible input parsing. [1] [2] [3] [4] [5] [6] [7] [8]
• The corresponding compareDatawithMC_qsub.sh script now passes additional parameters related to the reconstruction method and cut values and supports new command-line arguments for downstream processing.

Documentation and Usability:

• The README.md and release_tests/montecarlo/mc_data_comparison/README.md files have been significantly expanded and clarified, including updated zenith angle bin definitions, step-by-step instructions, and more detailed prerequisites and output structure explanations. [1] [2]
• The release_tests/sources/ANALYSIS.v2dl5.Crab.sh script is now more robust, parameterized, and user-friendly, with improved environment variable handling and error checking.

Environment and Tooling Updates:

• The environment.yml file now explicitly lists python, numpy, and matplotlib as dependencies and adds a note for updating with micromamba. [1] [2]
• The .pre-commit-config.yaml is updated to use newer versions of isort and flake8. [1] [2]

Bug Fixes and Minor Cleanups:

• Fixed a hardwired CUT warning in the IRF plotting script and removed a redundant .gitignore entry. [1] [2]

Zenith Angle Binning Update:

• The zenith angle bins (SZE, MZE, LZE) have been redefined, and a new "BZE" bin is introduced for beyond-reasonable zenith angles, reflected in both documentation and code.

These changes collectively improve the reliability, maintainability, and usability of the MC/data comparison and release test workflows.Minor fixes:

• readme for Crab analysis
• warning for hardwired values.

[Copilot]

Pull request overview

This PR updates the release-test tooling and documentation around Crab analysis and MC/data comparison, and improves library-loading paths by adding EVNDISPSYS support for libVAnaSum.so.

Changes:

• Extend libVAnaSum.so discovery to include $EVNDISPSYS and update related macros/scripts/docs.
• Adjust Crab runlist generation logic (skip non-4-telescope runs; tweak elevation bin thresholds) and update runlists/docs.
• Add a new Python plotting utility for sensitivity diagnostics and include needed Conda dependencies.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
release_tests/utilities/parameters.C	Adds EVNDISPSYS as a library search path and changes the library-load mechanism.
release_tests/sources/Crab/runlist_releaseTestingV6.dat	Removes two runs from the V6 release-testing runlist.
release_tests/sources/Crab/runlist_generator_from_anasum_log.sh	Skips non-4-telescope runs and updates elevation bin thresholds.
release_tests/sources/Crab/README.md	Fixes the referenced script name for generating anasum products from runlists.
release_tests/sources/Crab/plot_skymaps.C	Adds R__LOAD_LIBRARY directive for libVAnaSum.so.
release_tests/sources/Crab/plot_sensitivity.C	Adds R__LOAD_LIBRARY directive for libVAnaSum.so.
release_tests/sources/Crab/plot_sensitivity_from_anasum_log.py	New Python utility to parse anasum logs and generate diagnostic plots.
release_tests/sources/Crab/plot_lightcurves.C	Adds R__LOAD_LIBRARY directive for libVAnaSum.so.
release_tests/sources/Crab/plot_energy_spectra.C	Adds R__LOAD_LIBRARY directive for libVAnaSum.so.
release_tests/sources/Crab/plot_all.sh	Removes unused pwd variable.
release_tests/montecarlo/mc_data_comparison/README.md	Expands documentation and clarifies inputs/outputs/workflow for MC/data comparison.
release_tests/montecarlo/mc_data_comparison/compareDatawithMC.sh	Minor wording/formatting fixes and a small numeric literal adjustment.
release_tests/montecarlo/mc_data_comparison/.gitignore	Removes local ignore entry (moved to repo root).
release_tests/montecarlo/irf_plotting/irf_plotting.sh	Adds a warning about hardwired CUT.
environment.yml	Adds Python + numpy/matplotlib dependencies for the new plotting script.
.gitignore	Adds tmpdir ignore at repo root.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot review overview

🔵 Needs a closer look

It changes scientific-selection and batch workflows with unresolved correctness issues and externally dependent behavior.

Review effort: Balanced
Findings: 1 · 1 · 2

Open (4)

• Preserve BDT cuts for standard simulation comparisons · New
• Propagate Condor-script generator failures · New
• Document required XGB reconstruction files · New
• Update PR description with behavioral changes · New

Resolved since last review (3)

• Same return-type issue here: ProcessLine returns Longptr_t, not int. Using int risks truncation and…
• gROOT->ProcessLine returns a ROOT Longptr_t; storing it in an int can truncate the return value on…
• Minor: the skip message has a typo/double punctuation ("run..skipping") and inconsistent…

Previously missed (4)

In code that hasn't changed since last review

Handle empty parsed records before plotting

release_tests/​sources/​Crab/​plot_sensitivity_from_anasum_log.py:111

Matched files can all be skipped when an epoch, rates, Crab time, or run values cannot be parsed. In that case epochs remains empty and the subsequent violin plots fail rather than reporting that no complete records were found; guard this case before plotting.

Check ProcessLine error status before caching library success

release_tests/​utilities/​parameters.C:60

ProcessLine returns the evaluated expression value, not its execution status, so result >= 0 can mark a failed library load as successful. Pass the error-code argument and only cache success when it is zero; otherwise callers can proceed with unavailable dictionaries.

This issue also appears on line 68 of the same file.

Classify epochs dynamically beyond 2026

release_tests/​sources/​Crab/​plot_sensitivity_from_anasum_log.py:176

The comment says >= 2020, but enumerating years only through 2026 makes every later epoch blue. Derive the classification from the year tokens so new releases retain the intended behavior.

Update header documentation for new elevation thresholds

release_tests/​sources/​Crab/​runlist_generator_from_anasum_log.sh:128

These new thresholds no longer match the script header, which still documents the old >50, 40-50, and <40 degree ranges. Update that header to describe the 60/45/35-degree elevation boundaries so generated run-list categories are not misinterpreted.