Repository navigation
SHA-256 digest capture and verification for all downloads (Provenance Phase 1) #1296
Description
Activity
- added a commit that references this issue
on Aug 6, 2026 - added a commit that references this issue
on Aug 31, 2026 I propose breaking the provenance work into the following phases:
Phase 1 — Capture build lineage
Focus on answering: “Which upstream bytes, source changes, downstream sdist, and wheel produced this build?”
- Record upstream archive URL and digest.
- For Git sources, record repository, requested ref/tag, resolved commit, and source-tree identity.
- Record unpacking, repacking, patches, overrides, vendoring, plugin changes, and PKG-INFO status.
- Record normalized build context such as variant, platform, interpreter, and relevant configuration.
- Record downstream sdist and wheel digests.
- Link the artifacts as upstream input → downstream sdist → wheel.
- Write the complete lineage to provenance-index.json.
- Add lightweight artifact and lineage references to graph.json and build-order.json.
Phase 1 should preserve existing build behavior and plugin APIs. It should record current repacking and stub PKG-INFO behavior rather than changing them.
Phase 2 — Verify and safely reuse artifacts
- Track expected and observed digests separately.
- Verify downloads, existing files, and cache entries.
- Add continuity checks for previous build metadata.
- Reject conflicting digest claims.
- Make downstream sdist reuse depend on matching digest, source changes, and build context.
- Fail closed on integrity mismatches.
Phase 3 — Integrate downstream consumers
- Update builder onboarding to preserve upstream identity.
- Model source-mirror uploads as publications rather than replacements for the upstream artifact.
- Add optional upstream evidence when already available, such as publisher, repository, workflow, ref, commit, release date, wheel tags, or attestation presence.
- Do not fetch or verify attestations in this phase.
Future work — Improve sdist quality
The following should be handled separately because they change build behavior:
- avoid unnecessary sdist repacking where possible;
- generate correct PKG-INFO for Git and raw-tarball sources; and
- make sdist generation deterministic across variants and environments where practical.
This phased approach keeps the primary objective focused on recording complete upstream-to-downstream provenance while allowing verification, integration, and sdist-generation improvements to evolve independently.
- added 4 commits that reference this issue
on Sep 1, 2026
Summary
fromager downloads sdists and wheels without computing or verifying any cryptographic hash. A compromised upstream can replace artifact bytes between runs and fromager will accept them silently. This is Phase 1 of #1238.
Current state
download_url()streams bytes to disk with nohashlibcomputation#sha256=URL fragments from PyPI indexes are carried in URL strings but never verifiedpypi_simple.DistributionPackage.digestsis available but never readCandidate,DependencyNode, and build-order entries have no digest fieldProposed changes
download_url(), the single chokepoint for all HTTP downloads; return the digest alongside the file path#sha256=from download URLs and hard-fail on mismatch with computed digestdp.digestsfrompypi_simpleas a cross-check when creatingCandidateobjectsdigestfield toCandidate,DependencyNode/DependencyNodeDict, and build-order entries (backward-compatible viaNotRequired/defaults)provenance-index.json— aggregate per-package digests duringfinalize(), alongsidebuild-order.json