PR Profiling: Add link for diff of IB and PR Vtune reports and add script to generate page with the diff of IB and PR sorted-CPU-RES reports by gartung · Pull Request #2787 · cms-sw/cms-bot

gartung · 2026-06-10T14:47:31Z

No description provided.

cmsbuild · 2026-06-10T14:48:00Z

A new Pull Request was created by @gartung for branch master.

@akritkbehera, @cmsbuild, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

cmsbuild · 2026-06-10T14:48:01Z

cms-bot internal usage

gartung · 2026-06-10T15:12:10Z

enable profiling

cmsbuild · 2026-06-10T18:40:33Z

Pull request #2787 was updated.

cmsbuild · 2026-06-10T18:47:05Z

Pull request #2787 was updated.

cmsbuild · 2026-06-10T20:47:15Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3d44c3/53842/summary.html
COMMIT: 7628af1
CMSSW: CMSSW_20_1_X_2026-06-10-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2787/53842/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3248390
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3248372
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 191 log files, 159 edm output root files, 44 DQM output files

makortel · 2026-06-10T20:53:54Z

I see links to the IB and PR VTune reports but not the diff

How about moving the VTune link between the FastTimerService and ModuleAllocMonitor? Then the time-based tools would be next to each other.

gartung · 2026-06-10T21:17:10Z

Need to wait for the next IB to generate the vtune top-down json file before adding the comparison scripting.

cmsbuild · 2026-06-10T21:24:13Z

Pull request #2787 was updated.

cmsbuild · 2026-06-10T21:25:42Z

Pull request #2787 was updated.

cmsbuild · 2026-06-10T23:28:04Z

Pull request #2787 was updated.

cmsbuild · 2026-06-11T01:17:01Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3d44c3/53846/summary.html
COMMIT: 76731da
CMSSW: CMSSW_20_1_X_2026-06-10-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2787/53846/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3248390
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3248372
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 191 log files, 159 edm output root files, 44 DQM output files

…yloads

…filtering options

…tune event loop comparison

…rts.

… top-down reports

…t Loop Compare Viewer.

cmsbuild · 2026-06-16T14:56:15Z

Pull request #2787 was updated.

cmsbuild · 2026-06-16T16:51:16Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3d44c3/53985/summary.html
COMMIT: 618fc43
CMSSW: CMSSW_20_1_X_2026-06-15-2300/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2787/53985/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3414477
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3414456
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files

gartung · 2026-06-16T18:10:54Z

@makortel I generalized the script that generates the doEvent summary to include all produce functions. I had copilot look at all produce functions and find the parent function. It found that "EventSetup Producers" were not included in the parent function list. This should lead to no missing produce functions in the diff report unless a new producer was added/removed in the PR.

cmsbuild · 2026-06-16T20:04:07Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3d44c3/53992/summary.html
COMMIT: 618fc43
CMSSW: CMSSW_20_1_X_2026-06-15-2300/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2787/53992/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3414477
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3414456
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files

gartung · 2026-06-16T22:38:14Z

please test

cmsbuild · 2026-06-17T00:22:43Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3d44c3/53998/summary.html
COMMIT: 618fc43
CMSSW: CMSSW_20_1_X_2026-06-16-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2787/53998/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3414477
DQMHistoTests: Total failures: 28
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3414431
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files

makortel · 2026-06-17T07:16:32Z

I think the EventSetup producers should not be added in "RECO event loop". The EventSetup producers are not run on every event, but on IOV boundaries (which is likely once per job in all test jobs).

gartung · 2026-06-17T12:56:00Z

OK. Adding them did not make the number of IB vs PR functions reported the same. I will remove them from the report.

cmsbuild added orp-pending pending-signatures externals-pending tests-pending labels Jun 10, 2026

gartung linked an issue Jun 10, 2026 that may be closed by this pull request

Add diff table to VTune report cms-sw/framework-team#2248

Open

cmsbuild added tests-started tests-pending and removed tests-pending tests-started labels Jun 10, 2026

cmsbuild added tests-started and removed tests-pending labels Jun 10, 2026

cmsbuild added tests-approved and removed tests-started labels Jun 10, 2026

cmsbuild added tests-pending and removed tests-approved labels Jun 10, 2026

cmsbuild added tests-started and removed tests-pending labels Jun 10, 2026

cmsbuild removed the tests-started label Jun 11, 2026

gartung added 22 commits June 16, 2026 09:51

Add top-down comparison script to generate HTML with embedded JSON pa…

3fd0747

…yloads

Implement new feature for user authentication and improve error handling

e9a9bc2

Enhance top-down comparison viewer with expandable rows and improved …

73b3948

…filtering options

Update top-down compare viewer title and adjust sorting options for V…

0e05d0c

…tune event loop comparison

Put Vtune report between FastTimerService and ModuleAllocMonitor repo…

1085e64

…rts.

Add generation of Event Loop reports and adding links in summary table.

323a88b

Fix cell and link formatting.

849d28d

Use correct path for sorted_RES_CPU_step[0-9].{json,html}

d3e18e6

Remove leading slash from second file path in vtune comparison url

7292449

Fix links in profiling report to use correct paths for artifacts

715c383

When using full path use rel=noreferrer

89dd591

Remove rel=noreferrer add leading / to url

d1699c8

Put IB files in PROFILING_WORKFLOW directory.

6ae126e

Add executable bit

55da53a

Fix resource diff path in profiling workflow script

ad83456

Put the download location back to original. Try download location for…

852466f

… top-down reports

Output the top-down diff directly to upload directory

98d08bf

Fix link

ef159da

Fix the url not the target

94a2021

Improve column headers for clarity in top-down compare viewer

76e54f2

Update subheading for clarity and adjust filter options in Vtune Even…

5729cac

…t Loop Compare Viewer.

Get all *.gz files so top-down.csv.gz is saved

618fc43

Conversation

gartung commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gartung commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 10, 2026

Comparison Summary

Uh oh!

makortel commented Jun 10, 2026

Uh oh!

gartung commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 10, 2026

Uh oh!

cmsbuild commented Jun 11, 2026

Comparison Summary

Uh oh!

cmsbuild commented Jun 16, 2026

Uh oh!

cmsbuild commented Jun 16, 2026

Comparison Summary

Uh oh!

gartung commented Jun 16, 2026

Uh oh!

cmsbuild commented Jun 16, 2026

Comparison Summary

Uh oh!

gartung commented Jun 16, 2026

Uh oh!

cmsbuild commented Jun 17, 2026

Comparison Summary

Uh oh!

makortel commented Jun 17, 2026

Uh oh!

gartung commented Jun 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cmsbuild commented Jun 10, 2026 •

edited

Loading