Build and test conda packages and wheels by jharlow-intel · Pull Request #204 · IntelPython/mkl_random

jharlow-intel · 2026-10-23T20:23:40Z

Adds ci/ and .github/workflows/intel-ci.yml, which build the conda packages for Linux, Windows and Intel macOS and the wheels for Linux and Windows, on every declared Python, and test each in a clean environment.

conda-recipe/ is now the only recipe: the conda-forge one that was conda-recipe-cf/. The Intel-channel recipe and conda-package.yml are removed; conda-package-cf.yml builds conda-recipe/ and keeps its check names.
The conda packages build Python 3.10 to 3.15 plus 3.14t, from the python: list in conda_build_config.yaml, which also sets numpy: '2', as conda-build otherwise falls back to NumPy 1.26. The wheels follow the pyproject.toml classifiers: 3.10 to 3.15 plus 3.14t and 3.15t.
Wheels build with cibuildwheel, with MSVC on Windows, leave MKL out and require mkl >=2026.2,<2027 through [tool.gh-ci] wheel-requires, which ci/build.py adds to each wheel.
meson.build adds Library to CMake's search path on Windows, where mkl-devel puts its CMake config.
ci/test.py runs pytest --pyargs mkl_random on the installed package. The tests' hashlib.md5 calls pass usedforsecurity=False.
ci/benchmark.py runs the asv suite in benchmarks/ against the installed package: one quick sample of each benchmark on a pull request, a full measurement on a tag.

Miniforge is pinned to 26.7.2-0, as 26.9.4-0 makes setup-miniconda's conda init fail on Windows runners.

Overlap with #202. It changes conda-recipe-cf/, which this moves to conda-recipe/, so whichever lands second needs a rebase.

Validation: run 42099114121 built and tested the conda packages for 3.10 and 3.15 on Linux, Windows and macOS, and the wheels for 3.10 and 3.15 on Linux and Windows; all 21 jobs passed. Not run yet: the full matrix, which only master and tags build.

antonwolfy · 2026-10-26T21:29:32Z

+
+concurrency:
+  group: intel-ci-${{ github.ref }}
+  cancel-in-progress: true


With cancel-in-progress: true keyed on github.ref, two quick merges cancel the first commit's whole run. Jenkins, which waits for that run, gets no packages for it.

Suggested change

cancel-in-progress: true

cancel-in-progress: ${{ github.event_name == 'pull_request' }}

Applied in 6006c16, so a push to master no longer stops the run before it. Every tag is its own ref, so tag runs never cancelled each other either.

antonwolfy · 2026-10-26T21:37:08Z

+numpy:
+    - '2'


That only will work when ##202 is merged

Agreed, this should land after #202. With the current pin_compatible('numpy', min_pin="x.x"), the 3.15 package built here requires numpy >=2.5,<3.0a0 at run time, which #202 fixes. Once #202 merges I will rebase: numpy: '2' will come from #202, and 3.10 will leave the python: list along with its matrix.

antonwolfy · 2026-10-26T21:45:27Z

@@ -1,3 +1,16 @@
+# The Python versions to build, one per line and without selectors: ci/matrix.py
+# reads them for .github/workflows/intel-ci.yml. NumPy 2, as in conda-forge's
+# global pinning; conda-package-cf.yml passes its own with --python and --numpy.


--numpy is dropping in #202

Removed from the comment in 6006c16, so it stays true after #202.

antonwolfy · 2026-10-26T21:47:12Z

+        "--artifacts", required=True, type=Path, help="the downloaded artifact"
+    )
+    parser.add_argument(
+        "--work", required=True, type=Path, help="an empty scratch directory"


hooks.py doesn't check that --work is empty, and may install conda-index into the user's base environment.

Fixed in b758764. hooks.py now refuses a --work that is not an empty directory, and it writes the local channel's repodata.json itself, so it installs nothing into base.

antonwolfy · 2026-10-26T21:48:47Z

+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"


Is there any issue to use the newer python?

No issue: the job only runs ci/matrix.py, which needs nothing beyond the standard library. Moved to 3.14 in 6006c16.

antonwolfy · 2026-10-26T21:53:59Z

+          activate-environment: ""
+          conda-remove-defaults: "true"
+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0


Alternatively we can drop setup-python and run with shell: bash -el {0} with base activated.

bash is available on Windows runners too, and setup-miniconda supports this shell there. The step bodies become plain python ci/build.py … and python ci/hooks.py …. The $CONDA path code and hooks.py's re-run of itself would then only matter for local use.

Done in 6006c16: build and test run in bash -el {0} with base active, without setup-python. On Windows the new steps work: conda-build installs, the package builds and the imports pass. This run is still red on Windows because conda-forge's pytest 9.2.0, published today, dropped colorama again, so the recipe's pip check fails (conda-forge/pytest-feedstock#208). I will re-run once a fixed build is out.

Re-ran after conda-forge published the fixed pytest build: the run is green on Linux and Windows, test jobs included.

antonwolfy · 2026-10-26T22:00:21Z

@@ -0,0 +1,135 @@
+# Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every


The same PR is open in three repos at once.
Have you considered moving common things (scripts, workflow) to some common repo like https://github.com/IntelPython/devops-tools ?
That would help to reduce the duplication.

Yes, we considered it. Duplicated code is a known consequence of the new CI's design, where each repository carries everything its CI needs, and the design's other benefits make up for it.

Build and test steps run in a bash login shell with conda base active, so they call ci/build.py and ci/hooks.py directly and need no setup-python. Pushes to the default branch no longer cancel each other. Same as IntelPython/mkl_random#204.

vchamarthi

Went through this on the branch. The PR description looks stale: conda-recipe-cf is deleted, macOS and the wheels are built too, and Jenkins is enabled in dry run. There are also no AGENTS.md notes in the diff. Also, conda-package.yml is deleted, so the Intel channel build is no longer tested anywhere. Details inline.

vchamarthi · 2026-11-04T19:17:50Z

      - wheel  >=0.41.3
    run:
      - python
+      - {{ pin_compatible('numpy', min_pin="x.x", max_pin="x") }}


pin_compatible('numpy', min_pin="x.x", max_pin="x") resolves to >=2.2,<3.0a0 / >=2.5,<3.0a0 in the build logs, depending on the build. That excludes numpy 2.0 to 2.4 at runtime for the first one, and the minimum follows whatever numpy was in the host env. Is that the intent? A fixed lower bound like the old >=1.26.4 would be easier to reason about.

vchamarthi · 2026-11-04T19:17:50Z

+
+  test:
+    name: Test ${{ matrix.name }} Python ${{ matrix.python }}
+    needs: [matrix, build]


needs: [matrix, build] means a macOS conda failure skips test and so blocks the release, same for the wheel job at :155. If macOS is meant to gate releases that is fine, just checking it is on purpose.

vchamarthi · 2026-11-04T19:17:50Z

+  # macOS runs on Intel runners, as MKL has no Apple silicon build; Jenkins ignores it.
+  build:
+    name: ${{ matrix.name }} Python ${{ matrix.python }}
+    needs: matrix


This builds conda-recipe on every push and PR, and I don't see conda-package-cf.yml removed, so the same package may be built twice. Worth deleting or limiting the old workflow if it is now redundant.

vchamarthi · 2026-11-04T19:17:50Z

+"""
+Intel Python CI (gh-ci) test for mkl_random.
+
+GitHub Actions, through ci/hooks.py, and Intel's Jenkins both run this with one


Nit: the docstring says the test runs against a conda package, but the wheel job uses it too. Might be worth saying so.

Adds a ci/ directory and .github/workflows/intel-ci.yml. The workflow builds conda-recipe/ with conda-build and the wheels with cibuildwheel, for every platform and Python version the recipe and pyproject.toml declare, and tests each package in a clean environment with ci/test.py. conda-recipe/ is now the only recipe, the conda-forge one that was conda-recipe-cf/; the Intel-channel recipe and conda-package.yml are removed. conda-package-cf.yml reports the same check names. conda_build_config.yaml now lists the Python versions, and numpy: '2', as conda-build otherwise defaults to 1.26. The wheels' run-time requirement on mkl comes from [tool.gh-ci] wheel-requires in pyproject.toml, which ci/build.py adds to each wheel: in [project] it would fail the recipe's pip check, as conda-forge's mkl has no pip metadata. The wheels build against mkl-devel >=2026.2,<2027, the range they require at run time, and the Linux wheels are tagged manylinux_2_28 only, like the mkl wheel. ci/build.py keeps only the $ORIGIN entries of each ELF file's RPATH and RUNPATH in a wheel, as meson-python keeps the isolated build env's absolute lib directory, and refuses a file that would be left with none. ci/hooks.py tests each wheel in a venv made from conda-forge's Python, as pip users install it. Every python: entry in conda_build_config.yaml names its ABI, which ci/matrix.py requires, and every push and tag run gets a concurrency group of its own.

conda-build reads x86 as any x86 CPU and has no osx64 selector, while rattler-build has x86 for 32-bit only and osx64 for Intel macOS, so ci/matrix.py now gives conda-build its own values for both and refuses a conda-build skip that uses osx64. ci/build.py keeps a search path entry only when it is $ORIGIN or a directory under it, names RPATH and RUNPATH entries apart, and rewrites the wheels in scratch, so a refused wheel never reaches --output.

jharlow-intel marked this pull request as ready for review October 23, 2026 20:42

jharlow-intel requested review from antonwolfy, ndgrigorian, vlad-perevezentsev and xaleryb as code owners October 23, 2026 20:42

antonwolfy added this to the 1.7.0 release milestone Oct 26, 2026

antonwolfy reviewed Oct 26, 2026

View reviewed changes

jharlow-intel self-assigned this Oct 28, 2026

jharlow-intel requested a review from antonwolfy October 28, 2026 18:31

jharlow-intel mentioned this pull request Oct 28, 2026

Build and test conda packages and wheels IntelPython/mkl-service#265

Open

jharlow-intel force-pushed the ci/intel-python-ci branch 9 times, most recently from 2ecff7b to 8b5d74a Compare November 3, 2026 18:27

vchamarthi reviewed Nov 4, 2026

View reviewed changes

jharlow-intel changed the title ~~Build and test conda-recipe-cf for Intel Python CI~~ Build and test conda packages and wheels Nov 5, 2026

jharlow-intel force-pushed the ci/intel-python-ci branch from 754b382 to f881d71 Compare November 6, 2026 17:14

	cancel-in-progress: true
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

		@@ -0,0 +1,135 @@
		# Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every

Conversation

jharlow-intel commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jharlow-intel Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jharlow-intel Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vchamarthi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jharlow-intel commented Oct 23, 2026 •

edited

Loading

jharlow-intel Oct 28, 2026 •

edited

Loading

jharlow-intel Oct 28, 2026 •

edited

Loading