Skip to content

Build and test conda packages and wheels - #265

Open
jharlow-intel wants to merge 4 commits into
masterfrom
ci/intel-python-ci
Open

jharlow-intel wants to merge 4 commits into
masterfrom
ci/intel-python-ci

Conversation

@jharlow-intel

@jharlow-intel jharlow-intel commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Adds ci/ and .github/workflows/intel-ci.yml, which build the conda packages for Linux, Windows and Intel macOS and the wheels for Linux and Windows, on every declared Python, and test each in a clean environment.

  • conda-recipe/ is now the only recipe: the conda-forge one that was conda-recipe-cf/. The Intel-channel recipe and conda-package.yml are removed; conda-package-cf.yml stays and keeps its check names. No workflow in this repository builds or tests against the Intel channel any more: conda-package-cf.yml and intel-ci.yml both use conda-forge only.
  • The conda packages and the wheels build Python 3.11 to 3.15 plus 3.14t and 3.15t: the conda list follows the feedstock, plus 3.15t, and the wheels follow the pyproject.toml classifiers, which drop 3.10 here. requires-python still allows 3.10.
  • Wheels build with cibuildwheel, with MSVC on Windows, and leave MKL out. They require mkl from the mkl-devel version the build used up to its next major, through [tool.gh-ci] pin-compatible, which ci/build.py stamps into each wheel. ci/build.py also keeps only the $ORIGIN RPATH and RUNPATH entries of the wheels' ELF files, dropping the isolated build env's absolute path. The Linux wheels are tagged manylinux_2_28, like the mkl wheel.
  • meson.build adds Library to CMake's search path on Windows, where mkl-devel puts its CMake config.
  • ci/test.py runs pytest --pyargs mkl on the installed package: a conda package in a conda-forge environment, a wheel in a venv made from conda-forge's Python, so mkl/_init_helper.py is what loads MKL on Windows.

Validation: run 42277684124 built and tested the conda packages for 3.11 and 3.15 on Linux, Windows and macOS, and the wheels for 3.11 and 3.15 on Linux and Windows. Not run yet: the full matrix, which only master and tags build.

@jharlow-intel jharlow-intel self-assigned this Oct 28, 2026
Copilot AI balanced review requested due to automatic review settings October 28, 2026 21:28

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The native-package pipeline requires end-to-end Linux/Windows CI validation of conda solving and runtime loading for 3.14t/3.15t, which remain unverified by standard-library-only checks.

No open issues 🎉

What changed in this PR

Adds GitHub Actions build and package testing for Intel Python CI, preparing mkl-service for Jenkins integration. Jenkins is not yet enabled for this repository.

Changes:

  • Derives Linux and Windows build matrices from the conda-forge recipe, selecting four builds for pull requests and all 16 for master pushes and tags.
  • Adds build and isolated package-test scripts, with pytest results written as JUnit reports.
  • Documents the integration and its configuration.
File Description
conda-recipe-cf/​conda_build_config.yaml Declares Python versions and ABI variants for matrix generation.
ci/​test.py Checks the interpreter and runs installed-package tests with JUnit output.
ci/​README.md Documents CI operation and local commands.
ci/​matrix.py Generates platform and Python matrices from build declarations.
ci/​hooks.py Installs artifacts into clean environments and runs package tests.
ci/​config.yml Defines the workflow, artifact naming, and test requirements.
ci/​build.py Builds packages for a selected Python variant and collects artifacts.
AGENTS.md Adds a pointer to Intel Python CI documentation.
.github/​workflows/​intel-ci.yml Orchestrates matrix generation, builds, artifact uploads, and tests.
.github/​AGENTS.md Documents the workflow and its recipe-derived matrix exception.

Review effort: Balanced

Comment thread conda-recipe-cf/conda_build_config.yaml Resolved
@jharlow-intel
jharlow-intel force-pushed the ci/intel-python-ci branch 2 times, most recently from ee0bbbf to eaaedc1 Compare October 30, 2026 14:43
@jharlow-intel jharlow-intel reopened this Oct 30, 2026
@jharlow-intel
jharlow-intel force-pushed the ci/intel-python-ci branch 4 times, most recently from 0db81d3 to f4c2344 Compare October 30, 2026 15:27
@jharlow-intel
jharlow-intel force-pushed the ci/intel-python-ci branch 2 times, most recently from 926e627 to c484738 Compare November 3, 2026 18:00

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gh-ci side looks good to me. Scripts match the other three repos, artifact names line up with ci/config.yml, actions are pinned.

The PR description is out of date though. It says the workflow builds conda-recipe-cf (deleted now), Linux and Windows only (macOS and wheels are built too), and that Jenkins isn't on for mkl-service (it's enabled in config/gh-ci.json, dry run). It also doesn't say conda-package.yml and the Intel-channel build are gone, which is the part reviewers need to see. Rest are inline.

Comment thread .github/workflows/intel-ci.yml Resolved
Comment thread .github/workflows/intel-ci.yml Resolved
Comment thread .github/workflows/intel-ci.yml Resolved
Comment thread conda-recipe/conda_build_config.yaml Resolved
Comment thread conda-recipe/AGENTS.md Resolved
Comment thread mkl/tests/AGENTS.md Resolved
Comment thread ci/test.py Resolved
@jharlow-intel jharlow-intel changed the title Build and test conda-recipe-cf for Intel Python CI Build and test conda packages and wheels for Intel Python CI Nov 4, 2026
@jharlow-intel

Copy link
Copy Markdown
Collaborator Author

Thanks @vchamarthi, I've rewritten the description and the title to match what the PR does now.

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@jharlow-intel jharlow-intel changed the title Build and test conda packages and wheels for Intel Python CI Build and test conda packages and wheels Nov 5, 2026
@jharlow-intel
jharlow-intel force-pushed the ci/intel-python-ci branch 2 times, most recently from 7bba6b0 to 94ceba3 Compare November 5, 2026 21:04

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks Great!

@antonwolfy antonwolfy added this to the 2.10.0 release milestone Nov 6, 2026

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few points inline, none of them blocking. One more for the description: with conda-package.yml and the Intel-channel recipe removed, no public workflow builds or tests against the Intel channel any more (intel-ci.yml uses conda-forge only too). Worth saying that outright.

Comment thread pyproject.toml Resolved
Comment thread ci/hooks.py Resolved
Comment thread ci/matrix.py Resolved
Comment thread ci/matrix.py Resolved
Comment thread AGENTS.md Resolved
Comment thread ci/build.py Resolved
Comment thread pyproject.toml Resolved
Comment thread pyproject.toml Resolved
Comment thread .github/workflows/intel-ci.yml Resolved
Comment thread ci/matrix.py Resolved
Comment thread meson.build Resolved
Adds a ci/ directory and .github/workflows/intel-ci.yml. The workflow builds
conda-recipe/ with conda-build and the wheels with cibuildwheel, for every
platform and Python version the recipe and pyproject.toml declare, and tests
each package in a clean environment with ci/test.py. The test jobs run for
every package that built, even when another build failed. conda-build is
pinned to 26.9.1.

conda-recipe/ is now the only recipe, the conda-forge one that was
conda-recipe-cf/; the Intel-channel recipe and conda-package.yml, which copied
wheels out through WHEELS_OUTPUT_FOLDER, are removed. conda-package-cf.yml
reports the same check names, so the required checks still run. The conda
packages and the wheels both start at Python 3.11.

The wheels' run-time requirement on mkl comes from [tool.gh-ci] wheel-requires
in pyproject.toml, which ci/build.py adds to each wheel: in [project] it would
fail the recipe's pip check, as conda-forge's mkl has no pip metadata.

The wheels build against mkl-devel >=2026.2,<2027, the range they require
at run time, and the Linux wheels are tagged manylinux_2_28 only, like the
mkl wheel. ci/build.py keeps only the $ORIGIN entries of each ELF file's
RPATH and RUNPATH in a wheel, as meson-python keeps the isolated build
env's absolute lib directory, and refuses a file that would be left with
none. ci/hooks.py tests each wheel in a venv made from conda-forge's
Python, as pip users install it. Every python: entry in
conda_build_config.yaml names its ABI, which ci/matrix.py requires, and
every push and tag run gets a concurrency group of its own.

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the latest push: everything from my last round except the dev upload is addressed, and I resolved those threads. A few small points inline, none of them blocking.

Comment thread ci/matrix.py Resolved
Comment thread ci/build.py Resolved
Comment thread ci/build.py Resolved
Comment thread ci/README.md Resolved
Comment thread pyproject.toml Resolved
Comment thread .github/workflows/intel-ci.yml Resolved
conda-build reads x86 as any x86 CPU and has no osx64 selector, while
rattler-build has x86 for 32-bit only and osx64 for Intel macOS, so
ci/matrix.py now gives conda-build its own values for both and refuses a
conda-build skip that uses osx64.

ci/build.py keeps a search path entry only when it is $ORIGIN or a
directory under it, names RPATH and RUNPATH entries apart, and rewrites
the wheels in scratch, so a refused wheel never reaches --output.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The native-wheel pipeline’s ELF and loader changes still need build-and-runtime validation for free-threaded 3.14t/3.15t ABIs, as the cited run covers only regular Python 3.11/3.15.

No open issues 🎉

Review effort: Balanced

[build-system] requires plain mkl-devel again. [tool.gh-ci] pin-compatible
maps mkl to mkl-devel, and ci/build.py requires mkl >= the mkl-devel version
python -m build installed for the build, < its next major, the way conda's
run_exports pin it, so nothing needs a bump for a new MKL release. The
version comes from the list python -m build logs, and a build that logs none
fails.

The repair commands exclude libmkl_rt.so.* and mkl_rt.*.dll, so a newer MKL
library name is still left out of the wheels.
Comment thread pyproject.toml
[tool.cibuildwheel]
build-verbosity = 1

[tool.cibuildwheel.linux]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Linux wheels build without hardening flags. I compared the py3.11 Linux wheel from this PR's Intel CI run with the 2.9.0 one from the internal Jenkins build. The Jenkins wheel came out of the conda build, so conda-forge's gcc gave it -fstack-protector-strong and -Wl,-z,relro -Wl,-z,now: its .so files have __stack_chk_fail and BIND_NOW. The new wheel is built by the manylinux image's gcc 14 with no flags set, and has neither: no stack protector, and only partial RELRO.

The conda packages don't need anything, as conda-forge's compiler activation already sets these. The Linux build log shows CFLAGS=... -fstack-protector-strong ..., CPPFLAGS=... -D_FORTIFY_SOURCE=2 ... and LDFLAGS=... -Wl,-z,relro -Wl,-z,now ..., and the new conda .so files have both. Windows is fine too: with MSVC's defaults the .pyd files have the same characteristics as the Jenkins ones (dynamic base, high-entropy VA, NX compat).

Could we pass the same subset to the wheel build? Something like this, which I haven't run through cibuildwheel:

Suggested change
[tool.cibuildwheel.linux]
[tool.cibuildwheel.linux]
# The hardening flags conda-forge's compilers set for the conda packages
environment = { CFLAGS = "-fstack-protector-strong", CPPFLAGS = "-D_FORTIFY_SOURCE=2", LDFLAGS = "-Wl,-z,relro -Wl,-z,now" }

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, confirmed: the 3.11 wheel from run 42277684124 had partial RELRO, no __stack_chk_fail and no fortified calls, while the conda package from the same run has all three. The flags are in as suggested, and ci/build.py now refuses a wheel with an ELF file that lacks full RELRO (GNU_RELRO plus BIND_NOW), so they can't silently stop applying. The stack protector has no per-file check, as a library without arrays legitimately has no canary. Run 42291879334 built the wheels with them: every .so in the 3.11, 3.15 and 3.15t Linux wheels has full RELRO and __stack_chk_fail, and _mklinit has fortified calls, as in the conda package.

Comment thread ci/build.py
"--output-folder",
str(scratch / "conda"),
"--override-channels",
*channel_args,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More a question than a request: do we need the conda packages to record their source commit, to stay aligned with the ones the internal Jenkins build makes?

Jenkins calls conda build ... --extra-meta remote_url=https://github.com/IntelPython/mkl-service sha=<commit>, so the 2.9.0 packages have this in info/about.json:

"extra": {"...": "...", "remote_url": "https://github.com/IntelPython/mkl-service", "sha": "6cc5780b3359482b47b5663dfe91a33048117213"}

The packages built here have only recipe-maintainers there, and their info/git is empty, as the recipe builds from a local path. So a package alone no longer says which commit it came from. 2.10.0dev0 with build number 37 means 37 commits after the tag, which doesn't pin one commit across branches. For a tag build the tag identifies it, and the wheels never recorded a commit, from Jenkins or from here.

If something downstream reads that sha, it would be a small addition here, only when the variables are set so local runs keep working:

"--extra-meta",
f"remote_url={os.environ['GITHUB_SERVER_URL']}/{os.environ['GITHUB_REPOSITORY']}",
f"sha={os.environ['GITHUB_SHA']}",

If nothing does, leaving it out is fine with me.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added: when GITHUB_SERVER_URL, GITHUB_REPOSITORY and GITHUB_SHA are set, conda-build gets --extra-meta remote_url=<server>/<repo> sha=<GITHUB_SHA>, and local runs are unchanged. On a pull request GITHUB_SHA is the merge commit the job checked out, which is what the package is built from. The packages from run 42291879334 have both in info/about.json.

Comment thread ci/matrix.py Outdated
Comment on lines +594 to +599
"""All of them, or for a pull request the oldest and newest regular ones."""
if event not in PULL_REQUEST_EVENTS:
return pythons
regular = [python for python in pythons if not python.version.endswith("t")]
regular = regular or pythons
return list(dict.fromkeys(regular[:1] + regular[-1:]))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A pull request builds no free-threaded package in this workflow: for_event keeps only the regular versions, so this PR's runs built 3.11 and 3.15, and 3.14t and 3.15t first build on a push to master or a tag.

For the conda packages the other workflows cover it: conda-package-cf.yml builds and tests 3.14t and 3.15t from the same recipe on every PR, on Linux, Windows and macOS, and build_pip.yml and the two Clang workflows build them from source on Linux. What no PR builds is the free-threaded wheels: cibuildwheel's cp314t and cp315t in the manylinux container and with MSVC on Windows, and their test in a venv with mkl from PyPI. All of this workflow's runs so far were pull requests, so no free-threaded wheel has been built yet, and on Windows the only free-threaded build a PR runs is the conda one.

Could a pull request also build the newest free-threaded version? With this, a PR gets 3.11, 3.15 and 3.15t on each platform, which adds two wheel builds and two wheel tests, plus three conda legs. I checked it with matrix.py --event pull_request, with and without --system wheel, but not in Actions:

Suggested change
"""All of them, or for a pull request the oldest and newest regular ones."""
if event not in PULL_REQUEST_EVENTS:
return pythons
regular = [python for python in pythons if not python.version.endswith("t")]
regular = regular or pythons
return list(dict.fromkeys(regular[:1] + regular[-1:]))
"""
All of them, or for a pull request the oldest and newest regular ones and
the newest free-threaded one.
"""
if event not in PULL_REQUEST_EVENTS:
return pythons
regular = [python for python in pythons if not python.version.endswith("t")]
threaded = [python for python in pythons if python.version.endswith("t")]
regular = regular or pythons
return list(dict.fromkeys(regular[:1] + regular[-1:] + threaded[-1:]))

ci/README.md would need the same in "A pull request into master". If matrix.py should stay the same as in the other repos that use it, a workflow_dispatch: trigger would at least let the full matrix run before a merge.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done as suggested, and ci/README.md says so: a pull request now builds 3.11, 3.15 and 3.15t. Run 42291879334 built and tested the cp315t wheels on Linux and Windows, along with the 3.15t conda packages on all three platforms.

Comment thread ci/build.py
Comment on lines +294 to +295
# Some Windows build backends write the metadata with CRLF.
eol = "\r\n" if "\r\n" in text else "\n"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: eol comes from the whole file, so a METADATA whose headers use LF but whose description has CRLF, such as a README checked out with CRLF and copied as it is, gets eol = "\r\n". Then partition("\r\n\r\n") finds no blank line, and the Requires-Dist line goes at the end of the description:

>>> with_requirements(b"Metadata-Version: 2.1\nName: mkl-service\n\n# README\r\ntext\r\n", ["mkl >=2026.2.0,<2027"])
b'Metadata-Version: 2.1\nName: mkl-service\n\n# README\r\ntext\r\nRequires-Dist: mkl >=2026.2.0,<2027\r\n'

The build still logs build: added Requires-Dist ..., and pip installs that wheel without mkl, so the first sign would be the wheel test failing to load MKL. meson-python writes the description with LF today (the Windows wheels' METADATA has no CRLF), so nothing hits it now.

Taking the line ending from the headers alone fixes it, and gives the same output as now for LF, for CRLF, and for CRLF headers with an LF description, with or without one:

Suggested change
# Some Windows build backends write the metadata with CRLF.
eol = "\r\n" if "\r\n" in text else "\n"
# Some Windows build backends write the metadata with CRLF. Take the line
# ending from the headers, as the description can keep the README's.
headers = re.split(r"\r?\n\r?\n", text, maxsplit=1)[0]
eol = "\r\n" if "\r\n" in headers else "\n"

Comment thread ci/README.md Outdated

The scripts need nothing beyond the Python standard library and the tools each one's docstring names. macOS packages build and test on Intel runners, as MKL has no Apple silicon build.

**A pull request into `master`.** Actions builds the oldest and the newest regular Python versions on Linux, Windows and macOS, as a conda package and, where `pyproject.toml` declares the platform, as a wheel, and tests each package.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: intel-ci.yml runs on every pull request, as its pull_request: trigger has no branches: filter, like the other workflows here. So a pull request into another branch gets the same reduced matrix.

Suggested change
**A pull request into `master`.** Actions builds the oldest and the newest regular Python versions on Linux, Windows and macOS, as a conda package and, where `pyproject.toml` declares the platform, as a wheel, and tests each package.
**A pull request.** Actions builds the oldest and the newest regular Python versions on Linux, Windows and macOS, as a conda package and, where `pyproject.toml` declares the platform, as a wheel, and tests each package.

name: mkl-service ${{ matrix.name }} Python ${{ matrix.python }}
path: ${{ runner.temp }}/output/*.conda
if-no-files-found: error
# Pull requests keep them 3 days, tags 30.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the expression gives 3 days to every run that isn't a tag, so packages from pushes to master are kept 3 days too, not only the pull requests' ones. The wheel upload below uses the same expression.

Suggested change
# Pull requests keep them 3 days, tags 30.
# Tags keep them 30 days, pull requests and pushes to master 3.

The Linux wheels build with the hardening flags conda-forge's compilers set
for the conda packages: -fstack-protector-strong, -D_FORTIFY_SOURCE=2 and
-Wl,-z,relro -Wl,-z,now. ci/build.py refuses a wheel whose ELF files lack
full RELRO, so the flags can't silently stop applying.

conda-build gets --extra-meta remote_url=... sha=... when GitHub Actions
sets GITHUB_SERVER_URL, GITHUB_REPOSITORY and GITHUB_SHA, so each conda
package records the commit it was built from in info/about.json.

A pull request also builds and tests the newest free-threaded version, so
the free-threaded wheels build before a merge.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants