Repository navigation
Build and test conda packages and wheels - #265
jharlow-intel wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
🔵 Needs a closer look
The native-package pipeline requires end-to-end Linux/Windows CI validation of conda solving and runtime loading for 3.14t/3.15t, which remain unverified by standard-library-only checks.
No open issues 🎉
What changed in this PR
Adds GitHub Actions build and package testing for Intel Python CI, preparing mkl-service for Jenkins integration. Jenkins is not yet enabled for this repository.
Changes:
- Derives Linux and Windows build matrices from the conda-forge recipe, selecting four builds for pull requests and all 16 for master pushes and tags.
- Adds build and isolated package-test scripts, with pytest results written as JUnit reports.
- Documents the integration and its configuration.
| File | Description |
|---|---|
conda-recipe-cf/conda_build_config.yaml |
Declares Python versions and ABI variants for matrix generation. |
ci/test.py |
Checks the interpreter and runs installed-package tests with JUnit output. |
ci/README.md |
Documents CI operation and local commands. |
ci/matrix.py |
Generates platform and Python matrices from build declarations. |
ci/hooks.py |
Installs artifacts into clean environments and runs package tests. |
ci/config.yml |
Defines the workflow, artifact naming, and test requirements. |
ci/build.py |
Builds packages for a selected Python variant and collects artifacts. |
AGENTS.md |
Adds a pointer to Intel Python CI documentation. |
.github/workflows/intel-ci.yml |
Orchestrates matrix generation, builds, artifact uploads, and tests. |
.github/AGENTS.md |
Documents the workflow and its recipe-derived matrix exception. |
Review effort: Balanced
ee0bbbf to
eaaedc1
Compare
0db81d3 to
f4c2344
Compare
926e627 to
c484738
Compare
vchamarthi
left a comment
There was a problem hiding this comment.
Gh-ci side looks good to me. Scripts match the other three repos, artifact names line up with ci/config.yml, actions are pinned.
The PR description is out of date though. It says the workflow builds conda-recipe-cf (deleted now), Linux and Windows only (macOS and wheels are built too), and that Jenkins isn't on for mkl-service (it's enabled in config/gh-ci.json, dry run). It also doesn't say conda-package.yml and the Intel-channel build are gone, which is the part reviewers need to see. Rest are inline.
|
Thanks @vchamarthi, I've rewritten the description and the title to match what the PR does now. |
7bba6b0 to
94ceba3
Compare
antonwolfy
left a comment
There was a problem hiding this comment.
A few points inline, none of them blocking. One more for the description: with conda-package.yml and the Intel-channel recipe removed, no public workflow builds or tests against the Intel channel any more (intel-ci.yml uses conda-forge only too). Worth saying that outright.
Adds a ci/ directory and .github/workflows/intel-ci.yml. The workflow builds conda-recipe/ with conda-build and the wheels with cibuildwheel, for every platform and Python version the recipe and pyproject.toml declare, and tests each package in a clean environment with ci/test.py. The test jobs run for every package that built, even when another build failed. conda-build is pinned to 26.9.1. conda-recipe/ is now the only recipe, the conda-forge one that was conda-recipe-cf/; the Intel-channel recipe and conda-package.yml, which copied wheels out through WHEELS_OUTPUT_FOLDER, are removed. conda-package-cf.yml reports the same check names, so the required checks still run. The conda packages and the wheels both start at Python 3.11. The wheels' run-time requirement on mkl comes from [tool.gh-ci] wheel-requires in pyproject.toml, which ci/build.py adds to each wheel: in [project] it would fail the recipe's pip check, as conda-forge's mkl has no pip metadata. The wheels build against mkl-devel >=2026.2,<2027, the range they require at run time, and the Linux wheels are tagged manylinux_2_28 only, like the mkl wheel. ci/build.py keeps only the $ORIGIN entries of each ELF file's RPATH and RUNPATH in a wheel, as meson-python keeps the isolated build env's absolute lib directory, and refuses a file that would be left with none. ci/hooks.py tests each wheel in a venv made from conda-forge's Python, as pip users install it. Every python: entry in conda_build_config.yaml names its ABI, which ci/matrix.py requires, and every push and tag run gets a concurrency group of its own.
0d201fc to
30f391e
Compare
antonwolfy
left a comment
There was a problem hiding this comment.
Re-reviewed the latest push: everything from my last round except the dev upload is addressed, and I resolved those threads. A few small points inline, none of them blocking.
conda-build reads x86 as any x86 CPU and has no osx64 selector, while rattler-build has x86 for 32-bit only and osx64 for Intel macOS, so ci/matrix.py now gives conda-build its own values for both and refuses a conda-build skip that uses osx64. ci/build.py keeps a search path entry only when it is $ORIGIN or a directory under it, names RPATH and RUNPATH entries apart, and rewrites the wheels in scratch, so a refused wheel never reaches --output.
[build-system] requires plain mkl-devel again. [tool.gh-ci] pin-compatible maps mkl to mkl-devel, and ci/build.py requires mkl >= the mkl-devel version python -m build installed for the build, < its next major, the way conda's run_exports pin it, so nothing needs a bump for a new MKL release. The version comes from the list python -m build logs, and a build that logs none fails. The repair commands exclude libmkl_rt.so.* and mkl_rt.*.dll, so a newer MKL library name is still left out of the wheels.
| [tool.cibuildwheel] | ||
| build-verbosity = 1 | ||
|
|
||
| [tool.cibuildwheel.linux] |
There was a problem hiding this comment.
The Linux wheels build without hardening flags. I compared the py3.11 Linux wheel from this PR's Intel CI run with the 2.9.0 one from the internal Jenkins build. The Jenkins wheel came out of the conda build, so conda-forge's gcc gave it -fstack-protector-strong and -Wl,-z,relro -Wl,-z,now: its .so files have __stack_chk_fail and BIND_NOW. The new wheel is built by the manylinux image's gcc 14 with no flags set, and has neither: no stack protector, and only partial RELRO.
The conda packages don't need anything, as conda-forge's compiler activation already sets these. The Linux build log shows CFLAGS=... -fstack-protector-strong ..., CPPFLAGS=... -D_FORTIFY_SOURCE=2 ... and LDFLAGS=... -Wl,-z,relro -Wl,-z,now ..., and the new conda .so files have both. Windows is fine too: with MSVC's defaults the .pyd files have the same characteristics as the Jenkins ones (dynamic base, high-entropy VA, NX compat).
Could we pass the same subset to the wheel build? Something like this, which I haven't run through cibuildwheel:
| [tool.cibuildwheel.linux] | |
| [tool.cibuildwheel.linux] | |
| # The hardening flags conda-forge's compilers set for the conda packages | |
| environment = { CFLAGS = "-fstack-protector-strong", CPPFLAGS = "-D_FORTIFY_SOURCE=2", LDFLAGS = "-Wl,-z,relro -Wl,-z,now" } |
There was a problem hiding this comment.
Thanks, confirmed: the 3.11 wheel from run 42277684124 had partial RELRO, no __stack_chk_fail and no fortified calls, while the conda package from the same run has all three. The flags are in as suggested, and ci/build.py now refuses a wheel with an ELF file that lacks full RELRO (GNU_RELRO plus BIND_NOW), so they can't silently stop applying. The stack protector has no per-file check, as a library without arrays legitimately has no canary. Run 42291879334 built the wheels with them: every .so in the 3.11, 3.15 and 3.15t Linux wheels has full RELRO and __stack_chk_fail, and _mklinit has fortified calls, as in the conda package.
| "--output-folder", | ||
| str(scratch / "conda"), | ||
| "--override-channels", | ||
| *channel_args, |
There was a problem hiding this comment.
More a question than a request: do we need the conda packages to record their source commit, to stay aligned with the ones the internal Jenkins build makes?
Jenkins calls conda build ... --extra-meta remote_url=https://github.com/IntelPython/mkl-service sha=<commit>, so the 2.9.0 packages have this in info/about.json:
"extra": {"...": "...", "remote_url": "https://github.com/IntelPython/mkl-service", "sha": "6cc5780b3359482b47b5663dfe91a33048117213"}The packages built here have only recipe-maintainers there, and their info/git is empty, as the recipe builds from a local path. So a package alone no longer says which commit it came from. 2.10.0dev0 with build number 37 means 37 commits after the tag, which doesn't pin one commit across branches. For a tag build the tag identifies it, and the wheels never recorded a commit, from Jenkins or from here.
If something downstream reads that sha, it would be a small addition here, only when the variables are set so local runs keep working:
"--extra-meta",
f"remote_url={os.environ['GITHUB_SERVER_URL']}/{os.environ['GITHUB_REPOSITORY']}",
f"sha={os.environ['GITHUB_SHA']}",If nothing does, leaving it out is fine with me.
There was a problem hiding this comment.
Added: when GITHUB_SERVER_URL, GITHUB_REPOSITORY and GITHUB_SHA are set, conda-build gets --extra-meta remote_url=<server>/<repo> sha=<GITHUB_SHA>, and local runs are unchanged. On a pull request GITHUB_SHA is the merge commit the job checked out, which is what the package is built from. The packages from run 42291879334 have both in info/about.json.
| """All of them, or for a pull request the oldest and newest regular ones.""" | ||
| if event not in PULL_REQUEST_EVENTS: | ||
| return pythons | ||
| regular = [python for python in pythons if not python.version.endswith("t")] | ||
| regular = regular or pythons | ||
| return list(dict.fromkeys(regular[:1] + regular[-1:])) |
There was a problem hiding this comment.
A pull request builds no free-threaded package in this workflow: for_event keeps only the regular versions, so this PR's runs built 3.11 and 3.15, and 3.14t and 3.15t first build on a push to master or a tag.
For the conda packages the other workflows cover it: conda-package-cf.yml builds and tests 3.14t and 3.15t from the same recipe on every PR, on Linux, Windows and macOS, and build_pip.yml and the two Clang workflows build them from source on Linux. What no PR builds is the free-threaded wheels: cibuildwheel's cp314t and cp315t in the manylinux container and with MSVC on Windows, and their test in a venv with mkl from PyPI. All of this workflow's runs so far were pull requests, so no free-threaded wheel has been built yet, and on Windows the only free-threaded build a PR runs is the conda one.
Could a pull request also build the newest free-threaded version? With this, a PR gets 3.11, 3.15 and 3.15t on each platform, which adds two wheel builds and two wheel tests, plus three conda legs. I checked it with matrix.py --event pull_request, with and without --system wheel, but not in Actions:
| """All of them, or for a pull request the oldest and newest regular ones.""" | |
| if event not in PULL_REQUEST_EVENTS: | |
| return pythons | |
| regular = [python for python in pythons if not python.version.endswith("t")] | |
| regular = regular or pythons | |
| return list(dict.fromkeys(regular[:1] + regular[-1:])) | |
| """ | |
| All of them, or for a pull request the oldest and newest regular ones and | |
| the newest free-threaded one. | |
| """ | |
| if event not in PULL_REQUEST_EVENTS: | |
| return pythons | |
| regular = [python for python in pythons if not python.version.endswith("t")] | |
| threaded = [python for python in pythons if python.version.endswith("t")] | |
| regular = regular or pythons | |
| return list(dict.fromkeys(regular[:1] + regular[-1:] + threaded[-1:])) |
ci/README.md would need the same in "A pull request into master". If matrix.py should stay the same as in the other repos that use it, a workflow_dispatch: trigger would at least let the full matrix run before a merge.
There was a problem hiding this comment.
Done as suggested, and ci/README.md says so: a pull request now builds 3.11, 3.15 and 3.15t. Run 42291879334 built and tested the cp315t wheels on Linux and Windows, along with the 3.15t conda packages on all three platforms.
| # Some Windows build backends write the metadata with CRLF. | ||
| eol = "\r\n" if "\r\n" in text else "\n" |
There was a problem hiding this comment.
Nit: eol comes from the whole file, so a METADATA whose headers use LF but whose description has CRLF, such as a README checked out with CRLF and copied as it is, gets eol = "\r\n". Then partition("\r\n\r\n") finds no blank line, and the Requires-Dist line goes at the end of the description:
>>> with_requirements(b"Metadata-Version: 2.1\nName: mkl-service\n\n# README\r\ntext\r\n", ["mkl >=2026.2.0,<2027"])
b'Metadata-Version: 2.1\nName: mkl-service\n\n# README\r\ntext\r\nRequires-Dist: mkl >=2026.2.0,<2027\r\n'The build still logs build: added Requires-Dist ..., and pip installs that wheel without mkl, so the first sign would be the wheel test failing to load MKL. meson-python writes the description with LF today (the Windows wheels' METADATA has no CRLF), so nothing hits it now.
Taking the line ending from the headers alone fixes it, and gives the same output as now for LF, for CRLF, and for CRLF headers with an LF description, with or without one:
| # Some Windows build backends write the metadata with CRLF. | |
| eol = "\r\n" if "\r\n" in text else "\n" | |
| # Some Windows build backends write the metadata with CRLF. Take the line | |
| # ending from the headers, as the description can keep the README's. | |
| headers = re.split(r"\r?\n\r?\n", text, maxsplit=1)[0] | |
| eol = "\r\n" if "\r\n" in headers else "\n" |
|
|
||
| The scripts need nothing beyond the Python standard library and the tools each one's docstring names. macOS packages build and test on Intel runners, as MKL has no Apple silicon build. | ||
|
|
||
| **A pull request into `master`.** Actions builds the oldest and the newest regular Python versions on Linux, Windows and macOS, as a conda package and, where `pyproject.toml` declares the platform, as a wheel, and tests each package. |
There was a problem hiding this comment.
Nit: intel-ci.yml runs on every pull request, as its pull_request: trigger has no branches: filter, like the other workflows here. So a pull request into another branch gets the same reduced matrix.
| **A pull request into `master`.** Actions builds the oldest and the newest regular Python versions on Linux, Windows and macOS, as a conda package and, where `pyproject.toml` declares the platform, as a wheel, and tests each package. | |
| **A pull request.** Actions builds the oldest and the newest regular Python versions on Linux, Windows and macOS, as a conda package and, where `pyproject.toml` declares the platform, as a wheel, and tests each package. |
| name: mkl-service ${{ matrix.name }} Python ${{ matrix.python }} | ||
| path: ${{ runner.temp }}/output/*.conda | ||
| if-no-files-found: error | ||
| # Pull requests keep them 3 days, tags 30. |
There was a problem hiding this comment.
Nit: the expression gives 3 days to every run that isn't a tag, so packages from pushes to master are kept 3 days too, not only the pull requests' ones. The wheel upload below uses the same expression.
| # Pull requests keep them 3 days, tags 30. | |
| # Tags keep them 30 days, pull requests and pushes to master 3. |
The Linux wheels build with the hardening flags conda-forge's compilers set for the conda packages: -fstack-protector-strong, -D_FORTIFY_SOURCE=2 and -Wl,-z,relro -Wl,-z,now. ci/build.py refuses a wheel whose ELF files lack full RELRO, so the flags can't silently stop applying. conda-build gets --extra-meta remote_url=... sha=... when GitHub Actions sets GITHUB_SERVER_URL, GITHUB_REPOSITORY and GITHUB_SHA, so each conda package records the commit it was built from in info/about.json. A pull request also builds and tests the newest free-threaded version, so the free-threaded wheels build before a merge.
Adds
ci/and.github/workflows/intel-ci.yml, which build the conda packages for Linux, Windows and Intel macOS and the wheels for Linux and Windows, on every declared Python, and test each in a clean environment.conda-recipe/is now the only recipe: the conda-forge one that wasconda-recipe-cf/. The Intel-channel recipe andconda-package.ymlare removed;conda-package-cf.ymlstays and keeps its check names. No workflow in this repository builds or tests against the Intel channel any more:conda-package-cf.ymlandintel-ci.ymlboth use conda-forge only.pyproject.tomlclassifiers, which drop 3.10 here.requires-pythonstill allows 3.10.mklfrom themkl-develversion the build used up to its next major, through[tool.gh-ci] pin-compatible, whichci/build.pystamps into each wheel.ci/build.pyalso keeps only the$ORIGINRPATH and RUNPATH entries of the wheels' ELF files, dropping the isolated build env's absolute path. The Linux wheels are taggedmanylinux_2_28, like themklwheel.meson.buildaddsLibraryto CMake's search path on Windows, where mkl-devel puts its CMake config.ci/test.pyrunspytest --pyargs mklon the installed package: a conda package in a conda-forge environment, a wheel in a venv made from conda-forge's Python, somkl/_init_helper.pyis what loads MKL on Windows.Validation: run 42277684124 built and tested the conda packages for 3.11 and 3.15 on Linux, Windows and macOS, and the wheels for 3.11 and 3.15 on Linux and Windows. Not run yet: the full matrix, which only
masterand tags build.