Skip to content

speed up by x1000#4289

Merged
osbornjd merged 4 commits into
sPHENIX-Collaboration:masterfrom
pinkenburg:SvtxTruthRecoTableEval-speedup
Jun 10, 2026
Merged

speed up by x1000#4289
osbornjd merged 4 commits into
sPHENIX-Collaboration:masterfrom
pinkenburg:SvtxTruthRecoTableEval-speedup

Conversation

@pinkenburg

@pinkenburg pinkenburg commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • [X ] New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work for users)
  • Requiring change in macros repository (Please provide links to the macros pull request in the last section)
  • I am a member of GitHub organization of sPHENIX Collaboration, EIC, or ECCE (contact Chris Pinkenburg to join)

What kind of change does this PR introduce? (Bug fix, feature, ...)

codex managed to speed up SvtxTruthRecoTableEval by x1000. The resulting root files are binary identical with the previous version. This needs to be reviewed

TODOs (if applicable)

Links to other PRs in macros and calibration repositories (if applicable)

Motivation and Context

SvtxTruthRecoTableEval builds mapping tables between simulation truth particles and reconstructed SVTX tracks for downstream evaluation. This PR consolidates truth→reco and reco→truth mapping into a single pass to drastically reduce runtime (~1000× reported) while preserving the ROOT output format.

Key Changes

  • Single-pass mapping: Replaces separate fillTruthMap()/fillRecoMap() with fillTruthRecoMaps(topNode, trackeval, verbosity) that populates m_truthMap and m_recoMap in one traversal of SvtxTrackMap.
  • Verbosity & eval stack: process_event caches Verbosity(), sets it on m_svtxevalstack and SvtxTrackEval (asserting the latter), and issues a single combined log message when verbose.
  • Momentum pre-filtering: Selects truth particles (primary vs full per m_scanForPrimaries) and filters by squared momentum against m_minMomentumTruthMap^2 to skip low-momentum particles early.
  • Cluster-based aggregation: Uses SvtxClusterEval::all_truth_particles() and per-seed cluster keys to count contributions per truth track id and build weighted mappings for reco→truth and aggregated truth→reco.
  • Fast-sim handling: Preserves special-case SvtxTrack_FastSim behavior (uses get_truth_track_id() and trackeval->get_nclusters_contribution()).
  • API/header update: Header now forward-declares SvtxTrackEval and declares fillTruthRecoMaps(...); removed separate fillTruthMap/fillRecoMap definitions from the .cc file and updated includes (added SvtxClusterEval).
  • Logging reduction: Eliminates per-map repeated logging in favor of a single combined message (verbosity gated).

Potential Risk Areas

  • Output equivalence: Author reports binary-identical ROOT files; reviewers must still validate with regression tests and event-sample comparisons to confirm semantic equivalence.
  • Fast-sim correctness: Fast-sim path preserved but verify weighting/cluster-count semantics against legacy behavior with fast-sim testcases.
  • Memory usage: The single-pass algorithm builds temporary per-event maps keyed by truth id — check memory use for high-multiplicity events.
  • Thread-safety/concurrency: No explicit thread-safety changes; consolidated temporary structures should be reviewed before running in multi-threaded processing.
  • Dependency stability: Changes rely on SvtxClusterEval::all_truth_particles() and seed-cluster iteration semantics; interface changes there could silently alter mapping results.

Possible Future Improvements

  • Cache cluster→truth results within the event to avoid repeated clustereval lookups.
  • Profile remaining hotspots (cluster evaluation, unordered_map/set operations) and consider parallelizing safe per-track work.
  • Add automated performance and regression tests (including fast-sim cases) to guard against regressions and ensure binary-identical outputs.
  • Consider memory-limited or streaming strategies for very high-multiplicity events.

Note: AI summaries can miss subtle details—please review cluster-matching, weight accumulation, and the fast-sim path carefully during code review.

@pinkenburg pinkenburg marked this pull request as draft June 5, 2026 19:12
@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 606d09c1-6679-4525-b4b7-e8f15b0c7aaa

📥 Commits

Reviewing files that changed from the base of the PR and between 1f53daa and 5d974c7.

📒 Files selected for processing (1)
  • simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc
🚧 Files skipped from review as they are similar to previous changes (1)
  • simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc

📝 Walkthrough

Walkthrough

SvtxTruthRecoTableEval consolidates separate fillTruthMap and fillRecoMap implementations into a unified fillTruthRecoMaps method. The refactoring adds explicit verbosity control, introduces SvtxTrackEval as a parameter, and combines truth particle momentum filtering with reco-to-truth cluster contribution mapping into a single coherent function.

Changes

Truth/Reco Map Consolidation

Layer / File(s) Summary
Method interface update
simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.h
Header adds forward declaration for SvtxTrackEval and replaces private fillTruthMap() and fillRecoMap() with a single fillTruthRecoMaps(PHCompositeNode *topNode, SvtxTrackEval *trackeval, int verbosity) method signature.
Integration in process_event
simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc
process_event caches Verbosity() locally, applies it to m_svtxevalstack and the obtained SvtxTrackEval (with assertion), adds unified logging, and calls fillTruthRecoMaps(...) replacing prior separate calls.
Unified truth/reco map consolidation
simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc
New fillTruthRecoMaps implementation consolidates filtering (selects PHG4TruthInfoContainer range, applies squared-momentum threshold), reco-to-truth mapping (iterates SvtxTrackMap, counts cluster contributions per truth track via SvtxClusterEval, handles fast-sim special case), and truth-map population (transfers selected truth mappings, marks processed). Updated includes add SvtxClusterEval.h and standard containers/utilities.

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Infer (1.2.0)
simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc

In file included from simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc:2:
simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.h:4:10: fatal error: 'fun4all/SubsysReco.h' file not found
4 | #include <fun4all/SubsysReco.h>
| ^~~~~~~~~~~~~~~~~~~~~~
1 error generated.
simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc:132:3-151:3: ERROR translating statement 'ForStmt'
Aborting translation of method 'SvtxTruthRecoTableEval::fillTruthRecoMaps' in file 'simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc': "Assert_failure src/clang/cAst_utils.ml:249:53"
Aborting translation of method 'SvtxTruthRecoTableEval::process_event' in file 'simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc': "Assert_failure src/clang/cAst_utils.ml:249:53"
Uncaught Internal Error: "Assert_failure src/clang/cAst_utils.ml:249:53"
Error backtrace:
Raised at ClangFrontend__CAst_utils.get_decl_from_typ_ptr in file "src/clang/cAst_utils.ml", line 249, characters 53-65
Called fro

... [truncated 2200 characters] ...

n file "src/clang/cTrans.ml", line 4782, characters 12-47
Re-raised at IStdlib__IExn.reraise_after in file "src/istd/IExn.ml" (inlined), line 13, characters 2-50
Called from ClangFrontend__CTrans.CTrans_funct.instruction_log.(fun) in file "src/clang/cTrans.ml", line 4784, characters 10-1023
Called from ClangFrontend__CTrans.CTrans_funct.instruction in file "src/clang/cTrans.ml" (inlined), line 4765, characters 38-71
Called from ClangFrontend__CTrans.CTrans_funct.exec_with_node_creation in file "src/clang/cTrans.ml" (inlined), line 104, characters 20-38
Called from ClangFrontend__CTrans.CTrans_funct.get_clang_stmt_trans in file "src/clang/cTrans.ml" (inlined), line 5395, characters 4-69
Called from ClangFrontend__CTrans.CTrans_funct.get_custom_stmt_trans in file "src/clang/cTrans.ml", line


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@pinkenburg pinkenburg assigned pinkenburg and osbornjd and unassigned pinkenburg Jun 5, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 73b0f85a-74e1-4e6f-902b-ed0758e5f21b

📥 Commits

Reviewing files that changed from the base of the PR and between 9128ec0 and 36cda70.

📒 Files selected for processing (2)
  • simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.cc
  • simulation/g4simulation/g4eval/SvtxTruthRecoTableEval.h

Comment on lines +216 to +218
PHG4Particle *g4particle = truthinfo->GetParticle(fastsim_track->get_truth_track_id());
const float clusCont = trackeval->get_nclusters_contribution(track, g4particle);
truthmap[clusCont].insert(g4particle->get_track_id());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check GetParticle return behavior and usage patterns
rg -nP --type=cpp -C3 'GetParticle\s*\(' -g '!*Test*' -g '!*test*' | head -100

Repository: sPHENIX-Collaboration/coresoftware

Length of output: 8412


Critical: add null-check for g4particle in fast-sim truth handling.

PHG4TruthInfoContainer::GetParticle(...) can return nullptr, but SvtxTruthRecoTableEval.cc dereferences g4particle at g4particle->get_track_id() (and passes it to trackeval->get_nclusters_contribution) without a guard. Add an if (g4particle) before using it.

🛡️ Suggested null-check
   if (fastsim_track)
   {
     // Preserve SvtxTrackEval::all_truth_particles fast-sim special case for reco->truth maps only.
     PHG4Particle *g4particle = truthinfo->GetParticle(fastsim_track->get_truth_track_id());
+    if (g4particle)
+    {
       const float clusCont = trackeval->get_nclusters_contribution(track, g4particle);
       truthmap[clusCont].insert(g4particle->get_track_id());
+    }
   }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
PHG4Particle *g4particle = truthinfo->GetParticle(fastsim_track->get_truth_track_id());
const float clusCont = trackeval->get_nclusters_contribution(track, g4particle);
truthmap[clusCont].insert(g4particle->get_track_id());
PHG4Particle *g4particle = truthinfo->GetParticle(fastsim_track->get_truth_track_id());
if (g4particle)
{
const float clusCont = trackeval->get_nclusters_contribution(track, g4particle);
truthmap[clusCont].insert(g4particle->get_track_id());
}

@sphenix-jenkins-ci

Copy link
Copy Markdown

Build & test report

Report for commit 36cda703fe1169a1b1541855ab78177182b021ea:
Jenkins passed


Automatically generated by sPHENIX Jenkins continuous integration
sPHENIX             jenkins.io

@sphenix-jenkins-ci

Copy link
Copy Markdown

Build & test report

Report for commit 1f53daa5b4994223cec26fe113e569bcf9fc518a:
Jenkins passed


Automatically generated by sPHENIX Jenkins continuous integration
sPHENIX             jenkins.io

@osbornjd osbornjd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this on the oo sims and with some default sims and inspected the output from the module with the current new build and with this PR, and the results look identical for a single event. Reading through the code it looks to do the same thing in just a much more clever way rather than going track by track and using the evaluation machinery. We may consider seeing what else codex can improve in the evaluation machinery because it all uses stl containers probably not in the most simplified way...


// only record particle above minimal momentum requirement.
if (momentum < m_minMomentumTruthMap)
if (momentum2 < minMomentumTruthMap2)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why codex decided to change this from pT to pT^2 but in principle I don't think it matters, other than being potentially confusing when reading the code later

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment in the code - to point this out. It does save us a slow sqrt operation which is likely the reason why codex implemented the p cut this way

@pinkenburg pinkenburg marked this pull request as ready for review June 9, 2026 17:59
@sphenix-jenkins-ci

Copy link
Copy Markdown

Build & test report

Report for commit 582a0ad5a02094e41dd1f24258b1967de087f37a:
Jenkins on fire


Automatically generated by sPHENIX Jenkins continuous integration
sPHENIX             jenkins.io

@sphenix-jenkins-ci

Copy link
Copy Markdown

Build & test report

Report for commit 5d974c7f4af486ec02d84f2aecec3f13a0f188f6:
Jenkins passed


Automatically generated by sPHENIX Jenkins continuous integration
sPHENIX             jenkins.io

@osbornjd osbornjd merged commit 166a9a5 into sPHENIX-Collaboration:master Jun 10, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants