Avoid race condition when conditionally starting Lumi begin stream by Dr15Jones · Pull Request #51280 · cms-sw/cmssw

Dr15Jones · 2026-06-22T19:34:28Z

PR description:

Problem was the atomic could be incremented after it was already decremented to 0 once. This lead to the endLumiAsync being called twice for the same Lumi.

This fixes a problem seen in the IBs.

PR validation:

I had a small test program that tended to fail after 10s of 1000s of Lumis. With this changes, I have several times run a job which processed more than a million Lumis without issue.

resolves cms-sw/framework-team#2316
resolves #51265

Dr15Jones · 2026-06-22T19:34:35Z

please test

cmsbuild · 2026-06-22T19:34:52Z

cms-bot internal usage

cmsbuild · 2026-06-22T19:36:28Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51280/49888

cmsbuild · 2026-06-22T19:36:57Z

A new Pull Request was created by @Dr15Jones for master.

It involves the following packages:

FWCore/Framework (core)

@Dr15Jones, @makortel, @smuzaffar can you please review it and eventually sign? Thanks.
@makortel, @wddgit this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Problem was the atomic could be incremented after it was already decremented to 0 once. This lead to the endLumiAsync being called twice for the same Lumi.

Dr15Jones · 2026-06-22T20:08:14Z

please test

cmsbuild · 2026-06-22T20:09:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51280/49890

cmsbuild · 2026-06-22T20:10:05Z

Pull request #51280 was updated. @Dr15Jones, @makortel, @smuzaffar can you please check and sign again.

makortel · 2026-06-22T20:17:38Z

Looks good to me

cmsbuild · 2026-06-22T22:08:08Z

+1

Size: This PR adds an extra 44KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7871f7/54183/summary.html
COMMIT: c6c416a
CMSSW: CMSSW_20_1_X_2026-06-22-1100/el9_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/51280/54183/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3414477
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3414459
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files

cmsbuild · 2026-06-24T21:02:34Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51280/49926

cmsbuild · 2026-06-24T21:02:59Z

Pull request #51280 was updated. @Dr15Jones, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2026-06-24T22:56:31Z

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7871f7/54257/summary.html
COMMIT: 17a56de
CMSSW: CMSSW_20_1_X_2026-06-24-1100/el9_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/51280/54257/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3414477
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3414459
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files
TriggerResults: no differences found

- guarantee that at least 1 stream processes a Lumi even if the Lumi has no Events

cmsbuild · 2026-06-25T14:29:58Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51280/49939

cmsbuild · 2026-06-25T14:30:24Z

Pull request #51280 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

Dr15Jones · 2026-06-25T14:32:55Z

please test

The test holds an Event from the first Lumi until the Event from the 3rd Lumi is started. This forces the system to be able to skip the streamBeginLumi for the 2nd Lumi.

cmsbuild · 2026-06-25T16:27:15Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51280/49943

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/test/BuildFile.xml modified in PR(s): Add support for setting process.options from the command line #50585

cmsbuild · 2026-06-25T16:27:47Z

Pull request #51280 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

cmsbuild added this to the CMSSW_20_1_X milestone Jun 22, 2026

cmsbuild added core-pending pending-signatures orp-pending tests-started code-checks-pending labels Jun 22, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 22, 2026

makortel reviewed Jun 22, 2026

View reviewed changes

Comment thread FWCore/Framework/src/EventProcessor.cc Outdated

Avoid race condition when conditionally starting Lumi begin stream

c6c416a

Problem was the atomic could be incremented after it was already decremented to 0 once. This lead to the endLumiAsync being called twice for the same Lumi.

Dr15Jones force-pushed the fixLumi branch from 748e570 to c6c416a Compare June 22, 2026 20:07

cmsbuild added tests-pending code-checks-pending and removed tests-started code-checks-approved labels Jun 22, 2026

cmsbuild added tests-started and removed tests-pending labels Jun 22, 2026

Dr15Jones mentioned this pull request Jun 22, 2026

Crash in edm::EventProcessor::clearLumiPrincipal() #51265

Open

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 22, 2026

cmsbuild removed the tests-started label Jun 22, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 24, 2026

cmsbuild added tests-approved and removed tests-started labels Jun 24, 2026

Allow end lumi to proceed even if not all streams started the Lumi

5bf2873

- guarantee that at least 1 stream processes a Lumi even if the Lumi has no Events

Dr15Jones force-pushed the fixLumi branch from 17a56de to 5bf2873 Compare June 25, 2026 14:28

cmsbuild added tests-pending code-checks-pending and removed tests-approved code-checks-approved labels Jun 25, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 25, 2026

cmsbuild added tests-started and removed tests-pending labels Jun 25, 2026

Add test for being able to continue when a Lumi is blocked

20bc03f

The test holds an Event from the first Lumi until the Event from the 3rd Lumi is started. This forces the system to be able to skip the streamBeginLumi for the 2nd Lumi.

cmsbuild added tests-pending code-checks-pending and removed tests-started code-checks-approved labels Jun 25, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 25, 2026

cmsbuild mentioned this pull request Jun 26, 2026

non concurrent ESSources may only be in 1 thread jobs #51332

Open

Uh oh!

Conversation

Dr15Jones commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR description:

PR validation:

Uh oh!

Dr15Jones commented Jun 22, 2026

Uh oh!

cmsbuild commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Jun 22, 2026

Uh oh!

cmsbuild commented Jun 22, 2026

Uh oh!

Uh oh!

Dr15Jones commented Jun 22, 2026

Uh oh!

cmsbuild commented Jun 22, 2026

Uh oh!

cmsbuild commented Jun 22, 2026

Uh oh!

makortel commented Jun 22, 2026

Uh oh!

cmsbuild commented Jun 22, 2026

Comparison Summary

Uh oh!

cmsbuild commented Jun 24, 2026

Uh oh!

cmsbuild commented Jun 24, 2026

Uh oh!

cmsbuild commented Jun 24, 2026

Comparison Summary

Uh oh!

cmsbuild commented Jun 25, 2026

Uh oh!

cmsbuild commented Jun 25, 2026

Uh oh!

Dr15Jones commented Jun 25, 2026

Uh oh!

cmsbuild commented Jun 25, 2026

Uh oh!

cmsbuild commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Dr15Jones commented Jun 22, 2026 •

edited

Loading

cmsbuild commented Jun 22, 2026 •

edited

Loading