Repository navigation
[Bug]: Build dependency installs can discard requirements from earlier phases #1353
Description
Environment
- Fromager: 0.97.0
- Python: 3.12.14
- OS / architecture: Fedora Linux 44
Bug description
Summary
Fromager installs build-system, build-backend, and build-sdist dependencies
in separate calls to BuildEnvironment.install(). Each call uses
uv pip install --upgrade with only that phase's explicit requirements and
the shared constraints. Earlier requirements are not preserved during later
resolution, allowing a compatible build backend to be replaced with a version
outside the source package's declared compatibility bounds.
This was discovered through Fondue's pyarrow plugin, but it also applies to
compatibility bounds declared directly in a package's build-system requirements.
Evidence
- Execution: Fondue's CUDA/UBI9 builder container, building pyarrow 15.0.2.
- Related downstream investigation: https://redhat.atlassian.net/browse/RHAI-3914
and https://redhat.atlassian.net/browse/RHAI-3618.
Fondue adds a dynamically detected setuptools<82 requirement to pyarrow's
build-system dependencies. A local bootstrap log shows:
-
The build-system install honors the cap and installs setuptools 81.0.0.
-
A later build-backend install requests
cython>=0.29.31and
setuptools_scm<8.0.0, without the earlier setuptools cap. -
setuptools-scm brings setuptools into transitive resolution. The shared
constraints contain exclusions but no applicable upper bound. -
uv explicitly replaces the installed version:
- setuptools==81.0.0 + setuptools==84.0.0 -
The final environment reports setuptools 84.0.0, and pyarrow's setup code
fails withModuleNotFoundError: No module named 'pkg_resources'.
Setuptools removed pkg_resources in 82.0.0, not 81.0.0:
https://setuptools.pypa.io/en/latest/history.html#v82-0-0
Expected behavior
Subsequent dependency installation must continue satisfying the package's
previously declared build requirements. A genuinely incompatible later
requirement should cause an explicit resolution conflict instead of silently
invalidating the build environment.
Suggested implementation for discussion
Have each BuildEnvironment retain explicit requirements from successful
installations. Resolve subsequent installs against the union of previously
installed requirements and newly requested requirements:
- First install: build-system requirements.
- Second install: build-system plus build-backend requirements.
- Third install: build-system plus build-backend plus build-sdist requirements.
Implementing this in BuildEnvironment.install() would cover both ordinary
build preparation and bootstrap callers. Keep accumulation local to each build
environment, update it only after successful installation, and preserve
multiple specifiers for a package rather than overwriting by package name.
Retain shared constraints and --upgrade: upgrades should remain possible
within the accumulated compatibility requirements.
Keep phase-specific dependency files and graph classifications separate from
installation accumulation. They describe requirement origins. Review error
reporting so a conflict identifies both the new phase and earlier bounds.
An alternative is a package-local constraints mechanism, provided it handles
extras, markers, and installation requirements correctly.
Removing --upgrade alone is insufficient: a later dependency can still force
a version change. Globally capping setuptools is not an acceptable downstream
solution. Fondue can temporarily repeat its dynamic cap in later dependency
hooks until an upstream solution is available.
Relationship to #1351 and PR #1352
This fix does not require PR #1352, which corrects sdist dependency discovery
for issue #1351. The build-backend installation alone can trigger the upgrade.
Accumulating installation requirements can be implemented and tested with
either the current or corrected sdist hook selection.
PR #1352 should still be merged independently because it determines which
sdist requirements are discovered. It does not preserve requirements across
installation phases, and therefore does not resolve this issue by itself.
Activity