Skip to content

ART-19543: Refactor extensions build to use dnf download instead of rpm-ostree compose#1945

Open
locriandev wants to merge 1 commit into
openshift:release-5.0from
locriandev:feature/extensions-dnf-download
Open

ART-19543: Refactor extensions build to use dnf download instead of rpm-ostree compose#1945
locriandev wants to merge 1 commit into
openshift:release-5.0from
locriandev:feature/extensions-dnf-download

Conversation

@locriandev

@locriandev locriandev commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Refactors the extensions build process to use dnf download directly instead of rpm-ostree compose extensions, improving flexibility and simplifying the build process for RHEL 10.2 and other OS streams.

Changes

Build Script Refactor (extensions/build.sh)

  • Replace rpm-ostree: Switched from rpm-ostree compose extensions to direct dnf download commands
  • Parameterized repos: Introduced YUM_REPO_NAMES build arg to specify repos at build time
  • Direct RPM downloads: Each extension now downloads RPMs directly with --resolve --alldeps
  • Architecture-aware: Handles arch-specific extensions (kernel-rt for x86_64, kernel-64k for aarch64)
  • Version pinning: Kernel-related packages pinned to installed kernel EVR to ensure compatibility

New Build Configuration Files

  • build-args-5.0-9.8.conf: Repo names for RHEL 9.8
  • build-args-5.0-10.2.conf: Repo names for RHEL 10.2
  • build-args-5.0-c10s.conf: Repo names for CentOS Stream 10

Containerfile Update

  • Added ARG YUM_REPO_NAMES=overridden to accept repo names as build argument
  • Maintains backward compatibility with secret.repo mount

Removed Files

  • Deleted YAML extension definitions (no longer needed with dnf approach):
    • extensions/rhel-9.8.yaml
    • extensions/rhel-10.2.yaml
    • extensions/centos-10.yaml

@locriandev locriandev changed the title Refactor extensions build to use dnf download instead of rpm-ostree compose ART-19543: Refactor extensions build to use dnf download instead of rpm-ostree compose Jun 24, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 24, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 24, 2026

Copy link
Copy Markdown

@locriandev: This pull request references ART-19543 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

Refactors the extensions build process to use dnf download directly instead of rpm-ostree compose extensions, improving flexibility and simplifying the build process for RHEL 10.2 and other OS streams.

Changes

Build Script Refactor (extensions/build.sh)

  • Replace rpm-ostree: Switched from rpm-ostree compose extensions to direct dnf download commands
  • Parameterized repos: Introduced YUM_REPO_NAMES build arg to specify repos at build time
  • Direct RPM downloads: Each extension now downloads RPMs directly with --resolve --alldeps
  • Architecture-aware: Handles arch-specific extensions (kernel-rt for x86_64, kernel-64k for aarch64)
  • Version pinning: Kernel-related packages pinned to installed kernel EVR to ensure compatibility

New Build Configuration Files

  • build-args-5.0-9.8.conf: Repo names for RHEL 9.8
  • build-args-5.0-10.2.conf: Repo names for RHEL 10.2
  • build-args-5.0-c10s.conf: Repo names for CentOS Stream 10

Containerfile Update

  • Added ARG YUM_REPO_NAMES=overridden to accept repo names as build argument
  • Maintains backward compatibility with secret.repo mount

Removed Files

  • Deleted YAML extension definitions (no longer needed with dnf approach):
  • extensions/rhel-9.8.yaml
  • extensions/rhel-10.2.yaml
  • extensions/centos-10.yaml

Extensions Included

All existing extensions are preserved:

  • ipsec: libreswan, NetworkManager-libreswan, openvswitch3.5-ipsec
  • usbguard: usbguard package
  • kerberos: krb5-workstation, libkadm5
  • sysstat: sysstat package
  • kernel-devel: kernel-devel, kernel-headers (pinned to base kernel)
  • kernel: kernel, kernel-core, kernel-modules, kernel-modules-extra (development extension)
  • kernel-rt: Real-time kernel packages (x86_64 only)
  • kernel-64k: 64KB page size kernel (aarch64 only)
  • two-node-ha: pacemaker, pcs, fence-agents-all (RHEL only)

Benefits

  1. Simplified configuration: No need for complex YAML files with repo/package mappings
  2. Better flexibility: Easy to add/modify extensions without YAML schema knowledge
  3. Transparent process: Clear shell commands show exactly what's being downloaded
  4. RHEL 10 ready: Works seamlessly with RHEL 10.2 repos (no rpm-ostree compatibility issues)
  5. Explicit repo control: Build-time configuration via .conf files makes repo selection explicit

Related

  • Related to openshift-eng/ocp-build-data PR for RHEL 10.2 extensions image config
  • Based on "os · os-extensions-dnf-download" conversation

Test plan

  • Build extensions image for RHEL 9.8 with build-args-5.0-9.8.conf
  • Build extensions image for RHEL 10.2 with build-args-5.0-10.2.conf
  • Build extensions image for CentOS Stream 10 with build-args-5.0-c10s.conf
  • Verify all RPMs are downloaded to /usr/share/rpm-ostree/extensions/
  • Verify kernel packages match base image kernel version
  • Test on x86_64 (includes kernel-rt)
  • Test on aarch64 (includes kernel-64k)

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: locriandev
Once this PR has been reviewed and has the lgtm label, please assign jmarrero for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@travier

travier commented Jun 25, 2026

Copy link
Copy Markdown
Member

I would prefer to keep the split between the code/script doing the download and the configuration file with the extensions for each varsion/variant.

This is mentioned in:

Replace the OCP_VERSION placeholder with the actual OpenShift version.
This allows the same YAML file to be used across different OCP versions
(e.g. 4.23 and 5.0) without duplication.

The script could be a dnf download function with all the parameters and a loop over the entries in the YAML config.

@locriandev locriandev force-pushed the feature/extensions-dnf-download branch from c608390 to f9899d5 Compare June 25, 2026 17:30

@dustymabe dustymabe left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A step in the right direction. I can see a vision for how we can achieve the goal here cleanly.

Here's an initial round of review, but I'll have more questions as we go.

Comment thread extensions/build.sh Outdated
Comment on lines +34 to +37
packages=$(jq -r ".extensions[\"${extension_name}\"].packages[]" "$extensions_json" | while read -r pkg; do
# Perform variable substitution
eval "echo \"$pkg\""
done)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why we need the while loop here, but can probably fix it later.

Comment thread extensions/build.sh Outdated
Comment on lines +21 to +23
# Get kernel version for packages that need to match the base kernel
# Include epoch (0:) so dnf can disambiguate name from version in NEVRA format
kernel_evr=$(rpm -q --queryformat '%{VERSION}-%{RELEASE}' kernel)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should just versionlock everything in the node image and not worry about kernel_evr substitution here.

i.e I think a dnf versionlock should do it like we do for the node image:

os/build-node-image.sh

Lines 35 to 36 in d75a447

# Version lock to the specific packages installed on the system already
dnf --disablerepo=* versionlock add '*'

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread extensions/build.sh Outdated
# Download the packages
if [ -n "$packages" ]; then
echo "Downloading extension: ${extension_name}"
dnf --repo="${YUM_REPO_NAMES}" download --resolve --alldeps \

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we need --alldeps here.

When you do a build with this does glibc and glibc-devel get downloaded? or just glibc-devel?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

Comment thread extensions/build.sh Outdated
done)

# Download the packages
if [ -n "$packages" ]; then

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there were no packages to download we should error.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread extensions/build.sh Outdated
fi

# Download the extension packages
download_extension "$extension"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would argue that we should only call dnf download once instead of multiple times, but we can fix that later.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread extensions/build-args-5.0-10.2.conf Outdated
@@ -0,0 +1,2 @@
# The names of the yum repos to use for the extensions image build.
YUM_REPO_NAMES=rhel-10.2-baseos,rhel-10.2-appstream,rhel-10.2-server-ose-5.0,rhel-10.2-highavailability,rhel-10.2-fast-datapath,rhel-10.2-nfv

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since f6d3dba we use the top level build args file for extensions image builds too so let's not create a new build-args file here. Let's add this info there and use a different var name:

EXTENSIONS_YUM_REPO_NAMES=

?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED
@locriandev locriandev force-pushed the feature/extensions-dnf-download branch from f9899d5 to 7fb5588 Compare June 26, 2026 14:43
@openshift-ci

openshift-ci Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

@locriandev: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/images 7fb5588 link true /test images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants