Skip to content

CMake cleanups and RPM-build fixing#1018

Merged
alazzaro merged 46 commits into
cp2k:developfrom
Growl1234:cmake-libxs
Jun 11, 2026
Merged

CMake cleanups and RPM-build fixing#1018
alazzaro merged 46 commits into
cp2k:developfrom
Growl1234:cmake-libxs

Conversation

@Growl1234

@Growl1234 Growl1234 commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Motivation: cp2k/cp2k#5343 and cp2k/cp2k#5365

This PR makes the LIBXS/LIBXSMM/LIBXSTREAM handling more robust and less dependent on explicit ROOT variables or sibling source-tree layouts.

The dependency discovery now first tries pkg-config and standard CMake search paths, and only falls back to FetchContent when no local installation is found. This avoids unnecessary downloads when prebuilt libraries are already available. The *_ROOT variables are no longer used.

It also introduces stable wrapper targets for LIBXS and LIBXSTREAM, updates the OpenCL backend to use these targets and the detected LIBXSTREAM paths, and fixes the exported DBCSR CMake package so downstream users can resolve the required dependencies.

In addition, the default options are made more conservative: MPI, LIBXS, and LIBXSMM are no longer enabled by default. Users now need to enable them explicitly when needed, which avoids confusion in a default installation and follows the usual opt-in convention for optional dependencies. Also moves all option description to the # OPTIONS section.

The RPM-build is also fixed through: initializing tensor-test vectors directly to avoid GCC 16 nonnull warnings from initializer-list assignment that RPM-build treats as error.

@Growl1234 Growl1234 force-pushed the cmake-libxs branch 2 times, most recently from 60d295d to 9492d66 Compare June 6, 2026 20:30
@Growl1234 Growl1234 changed the title CMake: fix LIBXS/LIBXSTREAM discovery and revise options Fix libxs* discovery and revise options (CMake); fix RPM-build error Jun 6, 2026
@mtaillefumier

Copy link
Copy Markdown
Contributor

I support the intent; the implementation is questionable. Unless the dependency can be done with a pkg-config or a single CMake module, the entire detection should be in a file that is separated from CMakeLists.txt. What CMakeLists.txt and DBCSRConfig.cmake should do is only FetchContent or find_package.

@oschuett

oschuett commented Jun 7, 2026

Copy link
Copy Markdown
Member

I think, FetchContent in CMake should really be avoided as this is the responsibility of Spack/toolchain.

@Growl1234

Copy link
Copy Markdown
Contributor Author

Though I agree it's the right way forward to avoid FetchContent in cmake, I'm a bit hesitate here. @hfp what do you think?

@mtaillefumier

mtaillefumier commented Jun 7, 2026 via email

Copy link
Copy Markdown
Contributor

@Growl1234 Growl1234 marked this pull request as draft June 8, 2026 08:45
@hfp

hfp commented Jun 8, 2026

Copy link
Copy Markdown
Member

Fetching content is part of a chain and the last resort/fallback and it's only if the library in question is requested and not picked-up differently.

@mtaillefumier

Copy link
Copy Markdown
Contributor

Fetching content is part of a chain and the last resort/fallback and it's only if the library in question is requested and not picked-up differently.

True, the question is more do we want to do this or not. I agree with @oschuett that, while convenient, it mixes two different things, cmake as build system and cmake as package manager.

fetch_content does not provide any mechanism to manage compilations of the dependencies which is constraining in many cases. the functionality supports transitive builds (A requires B which requires C) only if all packages in the chain support cmake. Moreover it often requires some add_subdirectory and co to be fully functional.

So it is a great functionality for json, putty_xml or any small libraries that have no options and no dependencies. It becomes more difficult when other dependencies should be built as well.

@mtaillefumier

Copy link
Copy Markdown
Contributor

I will work on this PR for the next hours or so.

fetch_content is breaking already.

-- LIBXSTREAM not found locally -- downloading via FetchContent
CMake Error at build/_deps/libxstream-src/CMakeLists.txt:12 (find_package):
  Could not find a package configuration file provided by "libxs" with any of
  the following names:

    libxs.cps
    libxsConfig.cmake
    libxs-config.cmake

  Add the installation prefix of "libxs" to CMAKE_PREFIX_PATH or set
  "libxs_DIR" to a directory containing one of the above files.  If "libxs"
  provides a separate development package or SDK, be sure it has been
  installed.

@Growl1234

Copy link
Copy Markdown
Contributor Author

I will work on this PR for the next hours or so.

fetch_content is breaking already.

-- LIBXSTREAM not found locally -- downloading via FetchContent
CMake Error at build/_deps/libxstream-src/CMakeLists.txt:12 (find_package):
  Could not find a package configuration file provided by "libxs" with any of
  the following names:

    libxs.cps
    libxsConfig.cmake
    libxs-config.cmake

  Add the installation prefix of "libxs" to CMAKE_PREFIX_PATH or set
  "libxs_DIR" to a directory containing one of the above files.  If "libxs"
  provides a separate development package or SDK, be sure it has been
  installed.

I've added a FetchContent for standalone libxstream and will soon update package commit SHA.

@mtaillefumier

Copy link
Copy Markdown
Contributor

we can apply the same treatment here than on CP2K.

@Growl1234

Growl1234 commented Jun 8, 2026 via email

Copy link
Copy Markdown
Contributor Author

@Growl1234

Copy link
Copy Markdown
Contributor Author

And it seems the CMake path on the libxsmm side still needs to have a look...

I'll soon come back to this.

@hfp

hfp commented Jun 8, 2026

Copy link
Copy Markdown
Member

I will pull-in this PR or take over if you like? I had/have my changes in preparation as well.

@mtaillefumier

Copy link
Copy Markdown
Contributor

Could you make these changes available in a separate branch so wee can cherry pick accordingly. We should use cmake to build libxs and libxstream as it will remove two findpackage.cmake automatically.
What remain is libxsmm. if dbcsr rely on a version that support cmake then we should rely on it at as well. if not or for compatibility reason we want to have findLIBXSMM.cmake then I should know.

To be clear, do not use fetch_content inside a findPACKAGE.cmake. find_package(bla) is ok inside a findBLI.cmake (mirroring pkg_config usage)

@mtaillefumier

mtaillefumier commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

@hfp: I need your input on the dependency model of libxs and libxsmm, because their roles are still unclear:

  • Can dbcsr be built with libxsmm only?
  • Can dbcsr be built with libxs only?
  • When dbcsr is built with libxs, is libxsmm required or not?
  • Can the version of libxsmm used in dbcsr be built with cmake? Do we need a generic wrapper or not (for older versions of libxsmm)

@Growl1234 and I will take care of the CMake build system (I will, in any case). However being able to consult your own modifications will certainly help because we may miss important details.

@Growl1234

Copy link
Copy Markdown
Contributor Author

Thank you for taking care of CMake build system!

Removing FetchContent will break CI workflow, that's why I would like to keep it here temorarily. The workflow would have to be modified then.

Can the version of libxsmm used in dbcsr be built with cmake? Do we need a generic wrapper or not (for older versions of libxsmm)

You could have a look at libxsmm/libxsmm#1015 to see if improvements needed.

@Growl1234 Growl1234 force-pushed the cmake-libxs branch 2 times, most recently from 273f78e to 7be9fd3 Compare June 8, 2026 11:46
@hfp

hfp commented Jun 8, 2026

Copy link
Copy Markdown
Member

I am a bit puzzled about how this crosses. My idea would be to take those changes here and rework on top of mine. Once my PR is taken we can cooperatively revise it further. Would this work for you?

@Growl1234

Copy link
Copy Markdown
Contributor Author

I am a bit puzzled about how this crosses. My idea would be to take those changes here and rework on top of mine. Once my PR is taken we can cooperatively revise it further. Would this work for you?

Either way would work for me. Please feel free to proceed as you prefer.

@Growl1234 Growl1234 force-pushed the cmake-libxs branch 3 times, most recently from 232b86d to b72b8d5 Compare June 8, 2026 12:36
@Growl1234

Copy link
Copy Markdown
Contributor Author

It seems testings with libxsmm failed. @hfp could you have a look?

This failure looks like an ABI mismatch between libxs and libxsmm... again the concerns we talked before...

I will revert to hfp/libxs@89dc404 in GitHub CI for now, which remains the warning existing but avoids the runtime error.

@hfp

hfp commented Jun 9, 2026

Copy link
Copy Markdown
Member

It seems testings with libxsmm failed. @hfp could you have a look?

This failure looks like an ABI mismatch between libxs and libxsmm... again the concerns we talked before...

I will revert to hfp/libxs@89dc404 in GitHub CI for now, which remains the warning existing but avoids the runtime error.

Do you have the test's output? I can see your last change passed (this is current state like green):
04b1612
... which uses (by the time of writing)
LIBXSTREAM: e7bcb558ff1f3d1324f254ee1875ba81eadf353c (latest)
LIBXS: 89dc404a8fb5fd7e5549a032e452dbc60bbafbdf (2nd latest) -- however, this is newer than what it "reverted" and it passes!

If you change to LIBXS ccfb2b6069ba9bad033a550905cb032b9a7346f1 (latest), it should pass. I think you never tested this. I cannot find any commit that went through CI testing "ccfb2b6069ba9bad033a550905cb032b9a7346f1"?

@Growl1234

Growl1234 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Sorry, I did it through force-push so the results are not preserved here. I can reuse the latest commit. It passed on building but failed on testing after build

@Growl1234

Growl1234 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

It passes... so maybe a false positive?

Here's the previous message:
https://github.com/cp2k/dbcsr/actions/runs/27214743741/job/80353175083

job-logs(11).txt

@hfp

hfp commented Jun 10, 2026

Copy link
Copy Markdown
Member

It passes... so maybe a false positive?

Here's the previous message: https://github.com/cp2k/dbcsr/actions/runs/27214743741/job/80353175083

job-logs(11).txt

Thanks for sharing! If it a transient failure it points towards a race condition. There is nothing pending in this direction and things are thoroughly tested in LIBXS. It can be also on the DBCSR side. Once we have this PR merged, there will be at least one more to bring-in released version instead of SHAs.

Thanks for trying!

@hfp

hfp commented Jun 10, 2026

Copy link
Copy Markdown
Member

My broader concern is that LIBXS internal implementation details are being pushed into external projects: they effectively become de facto standards that then require modifications in AMD InfinityHub, Spack, and other packaging stacks. That is not sustainable for downstreams.

I am affiliated - true, but only to mention "MKL" as an optional thing (if it helps or if it's applicable - fine). In fact, I did everything to leave out those product stuff not to mention that using LIBXSMM kernels will help on Alps or ARM in general (I hope!). I am interested in foundational things to perhaps advance science. So, LIBXS will play-out for actual science going forward. I will have some talk at CP2K's 25th anniversary about it. I have no idea what "InfinityHub" is, and wrt Spack - I like the approach and I am contributing to it. I am sorry, it's not my expertise to fit your CMake taste. LLMs took over (update for you) and so it will with all this "CMake religion" and Spack too. This PR reworked DBCSR's CMake with pieces unrelated to my work. I only feel blamed for my contribution. I never expected my contribution to CP2K/DBCSR to be fully transactional or atomic. My and your nice work here in this PR is only normal like making this work and I expect it (not necessarily a "weekend call" ;-). Over the course of this PR (and the CP2K one), I was almost excluded from my responsibility to "fix the things I broke". That's unexpected to me. To wrap up the current/merged state of DBCSR develop: it passed all tests as one can still see. I knew the OpenCL test etc needed a 2nd PR but that was/is (still) true.

For me the core issue is that this integration is clearly still experimental and fragile, and it should never have been merged into a central dependency like DBCSR or CP2K without a period of broader testing in real build and packaging environments and one month before official release. Removing the old selection mechanism will force users to change their workflows. They already had difficulties with the cmake transition and we now introduce breaking changes whose repercussion is still unknown.

I think we do not stop science because of CMake taste (see above).

@Growl1234

Copy link
Copy Markdown
Contributor Author

I think, we are not intending to blame the libxs contribution or question its scientific value. The concern is only about the downstream integration contract. This is not "A vs B"; it's "A and B both matter"...

@mtaillefumier

Copy link
Copy Markdown
Contributor

This is not a matter of “CMake taste”. Modern CMake projects across the ecosystem follow broadly similar patterns because that is what enables maintainability, reproducibility, and straightforward reuse in downstream projects and package managers.

The PR in question was merged into develop without review and introduced changes that effectively require downstreams to mirror internal details of a dependency in order to keep existing builds working. Saying it is “standard practice” for a dependency to leak its internal implementation details into every project that uses it would need concrete evidence; my experience with CP2K, Spack and other HPC codes suggests the opposite.

A clean CMake build system is not an aesthetic preference. It has a very real maintenance cost and impact for everyone involved: developers, maintainers, vendors, and packagers. Poorly defined dependencies and ad‑hoc integration work translate directly into extra time spent fixing toolchains, CI, and containers instead of improving the code or the science.

Advancing science and following sound engineering practices are not in conflict, but they are also not the same thing. I also publish scientific work, so my concern here is not about slowing down science, but about basic engineering discipline around dependencies and releases so that a central library like DBCSR remains usable and stable for its downstream users

@mtaillefumier

Copy link
Copy Markdown
Contributor

I think, we are not intending to blame the libxs contribution or question its scientific value. The concern is only about the downstream integration contract. This is not "A vs B"; it's "A and B both matter"...

I fully agree with this.

@hfp

hfp commented Jun 10, 2026

Copy link
Copy Markdown
Member

I may send another PR regarding LIBXSMM and cleanup. It might be wise to still have a fallback to PkgConfig.

Comment thread src/cmake/DBCSRConfig.cmake.in

@alazzaro alazzaro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through most of the changes, thanks to all for cleaning up and making it work again!

@alazzaro alazzaro merged commit 1ba44db into cp2k:develop Jun 11, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants