Skip to content

METAL-1745: Add 5.0 jobs for rhcos10 worker nodes #79338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sgoveas wants to merge 1 commit into
base: main
Choose a base branch
from
Open

METAL-1745: Add 5.0 jobs for rhcos10 worker nodes #79338

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions ...openshift-tests-private/openshift-openshift-tests-private-release-5.0__multi-nightly.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3148,6 +3148,42 @@ tests:
test:
- chain: openshift-e2e-test-qe-destructive
workflow: cucushift-installer-rehearse-aws-ipi-custom-dns
- as: metal-ipi-ovn-ipv4-rhcos10-worker-tp-amd-f14
capabilities:
- intranet
cron: 6 18 13,27 * *
steps:
cluster_profile: equinix-ocp-metal-qe
env:
AUX_HOST: openshift-qe-metal-ci.arm.eng.rdu2.redhat.com
DISCONNECTED: "false"
FEATURE_SET: TechPreviewNoUpgrade
RESERVE_BOOTSTRAP: "false"
WORKER_COREOS_STREAM: rhel-10
architecture: amd64
masters: "3"
workers: "2"
test:
- chain: openshift-e2e-test-qe
workflow: baremetal-lab-ipi
- as: metal-ipi-ovn-ipv4-vmedia-rhcos10-worker-tp-arm-f14
capabilities:
- intranet
cron: 15 3 6,20 * *
steps:
cluster_profile: equinix-ocp-metal-qe
env:
AUX_HOST: openshift-qe-metal-ci.arm.eng.rdu2.redhat.com
DISCONNECTED: "false"
FEATURE_SET: TechPreviewNoUpgrade
RESERVE_BOOTSTRAP: "false"
WORKER_COREOS_STREAM: rhel-10
architecture: arm64
masters: "3"
workers: "2"
test:
- chain: openshift-e2e-test-qe
workflow: baremetal-lab-ipi-virtual-media
zz_generated_metadata:
branch: release-5.0
org: openshift
Expand Down
182 changes: 182 additions & 0 deletions ...hift/openshift-tests-private/openshift-openshift-tests-private-release-5.0-periodics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55693,6 +55693,97 @@ periodics:
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: build09
cron: 6 18 13,27 * *
decorate: true
decoration_config:
skip_cloning: true
extra_refs:
- base_ref: release-5.0
org: openshift
repo: openshift-tests-private
labels:
capability/intranet: intranet
ci-operator.openshift.io/cloud: equinix-ocp-metal
ci-operator.openshift.io/cloud-cluster-profile: equinix-ocp-metal-qe
ci-operator.openshift.io/variant: multi-nightly
ci.openshift.io/generator: prowgen
job-release: "5.0"
pj-rehearse.openshift.io/can-be-rehearsed: "true"
name: periodic-ci-openshift-openshift-tests-private-release-5.0-multi-nightly-metal-ipi-ovn-ipv4-rhcos10-worker-tp-amd-f14
spec:
containers:
- args:
- --gcs-upload-secret=/secrets/gcs/service-account.json
- --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson
- --lease-server-credentials-file=/etc/boskos/credentials
- --oauth-token-path=/usr/local/github-credentials/oauth
- --report-credentials-file=/etc/report/credentials
- --secret-dir=/secrets/ci-pull-credentials
- --target=metal-ipi-ovn-ipv4-rhcos10-worker-tp-amd-f14
- --variant=multi-nightly
command:
- ci-operator
env:
- name: HTTP_SERVER_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
image: quay-proxy.ci.openshift.org/openshift/ci:ci_ci-operator_latest
imagePullPolicy: Always
name: ""
ports:
- containerPort: 8080
name: http
resources:
requests:
cpu: 10m
volumeMounts:
- mountPath: /etc/boskos
name: boskos
readOnly: true
- mountPath: /secrets/ci-pull-credentials
name: ci-pull-credentials
readOnly: true
- mountPath: /secrets/gcs
name: gcs-credentials
readOnly: true
- mountPath: /usr/local/github-credentials
name: github-credentials-openshift-ci-robot-private-git-cloner
readOnly: true
- mountPath: /secrets/manifest-tool
name: manifest-tool-local-pusher
readOnly: true
- mountPath: /etc/pull-secret
name: pull-secret
readOnly: true
- mountPath: /etc/report
name: result-aggregator
readOnly: true
serviceAccountName: ci-operator
volumes:
- name: boskos
secret:
items:
- key: credentials
path: credentials
secretName: boskos-credentials
- name: ci-pull-credentials
secret:
secretName: ci-pull-credentials
- name: github-credentials-openshift-ci-robot-private-git-cloner
secret:
secretName: github-credentials-openshift-ci-robot-private-git-cloner
- name: manifest-tool-local-pusher
secret:
secretName: manifest-tool-local-pusher
- name: pull-secret
secret:
secretName: registry-pull-credentials
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: build09
cron: 20 14 3,10,17,24 * *
Expand Down Expand Up @@ -56057,6 +56148,97 @@ periodics:
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: build09
cron: 15 3 6,20 * *
decorate: true
decoration_config:
skip_cloning: true
extra_refs:
- base_ref: release-5.0
org: openshift
repo: openshift-tests-private
labels:
capability/intranet: intranet
ci-operator.openshift.io/cloud: equinix-ocp-metal
ci-operator.openshift.io/cloud-cluster-profile: equinix-ocp-metal-qe
ci-operator.openshift.io/variant: multi-nightly
ci.openshift.io/generator: prowgen
job-release: "5.0"
pj-rehearse.openshift.io/can-be-rehearsed: "true"
name: periodic-ci-openshift-openshift-tests-private-release-5.0-multi-nightly-metal-ipi-ovn-ipv4-vmedia-rhcos10-worker-tp-arm-f14
spec:
containers:
- args:
- --gcs-upload-secret=/secrets/gcs/service-account.json
- --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson
- --lease-server-credentials-file=/etc/boskos/credentials
- --oauth-token-path=/usr/local/github-credentials/oauth
- --report-credentials-file=/etc/report/credentials
- --secret-dir=/secrets/ci-pull-credentials
- --target=metal-ipi-ovn-ipv4-vmedia-rhcos10-worker-tp-arm-f14
- --variant=multi-nightly
command:
- ci-operator
env:
- name: HTTP_SERVER_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
image: quay-proxy.ci.openshift.org/openshift/ci:ci_ci-operator_latest
imagePullPolicy: Always
name: ""
ports:
- containerPort: 8080
name: http
resources:
requests:
cpu: 10m
volumeMounts:
- mountPath: /etc/boskos
name: boskos
readOnly: true
- mountPath: /secrets/ci-pull-credentials
name: ci-pull-credentials
readOnly: true
- mountPath: /secrets/gcs
name: gcs-credentials
readOnly: true
- mountPath: /usr/local/github-credentials
name: github-credentials-openshift-ci-robot-private-git-cloner
readOnly: true
- mountPath: /secrets/manifest-tool
name: manifest-tool-local-pusher
readOnly: true
- mountPath: /etc/pull-secret
name: pull-secret
readOnly: true
- mountPath: /etc/report
name: result-aggregator
readOnly: true
serviceAccountName: ci-operator
volumes:
- name: boskos
secret:
items:
- key: credentials
path: credentials
secretName: boskos-credentials
- name: ci-pull-credentials
secret:
secretName: ci-pull-credentials
- name: github-credentials-openshift-ci-robot-private-git-cloner
secret:
secretName: github-credentials-openshift-ci-robot-private-git-cloner
- name: manifest-tool-local-pusher
secret:
secretName: manifest-tool-local-pusher
- name: pull-secret
secret:
secretName: registry-pull-credentials
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: build09
cron: 11 14 1,8,15,22 * *
Expand Down
40 changes: 40 additions & 0 deletions ci-operator/step-registry/baremetal/lab/ipi/install/baremetal-lab-ipi-install-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,46 @@ if [[ "$ENABLE_CAPI" == "true" ]]; then
sed -i 's/watchAllNamespaces: false/watchAllNamespaces: true/' "${INSTALL_DIR}/openshift/99_baremetal-provisioning-config.yaml"
fi

# Patch worker BMH manifests to use the specified CoreOS stream.
# This controls which IPA (kernel/initrd/rootfs) version is used
# during PXE boot for worker nodes.
if [[ -n "${WORKER_COREOS_STREAM:-}" ]]; then
if [[ ! "${WORKER_COREOS_STREAM}" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "[ERROR] Invalid WORKER_COREOS_STREAM: ${WORKER_COREOS_STREAM}"
exit 1
fi
esc_stream="$(printf '%s' "${WORKER_COREOS_STREAM}" | sed 's/[\/&]/\\&/g')"
for bmh_file in "${INSTALL_DIR}"/openshift/99_openshift-cluster-api_hosts-*.yaml; do
if ! grep -q 'installer.openshift.io/role: control-plane' "${bmh_file}"; then
sed -i "s/coreos.openshift.io\/stream: .*/coreos.openshift.io\/stream: ${esc_stream}/" "${bmh_file}"
fi
done
# Patch worker MachineSet hostSelector to match the new stream
for ms_file in "${INSTALL_DIR}"/openshift/99_openshift-cluster-api_worker-machineset-*.yaml; do
sed -i "s/coreos.openshift.io\/stream: .*/coreos.openshift.io\/stream: ${WORKER_COREOS_STREAM}/" "${ms_file}"
done
# Set the worker MachineConfigPool to use the specified stream for
# the on-disk OS. Requires the OSStreams feature gate (TechPreviewNoUpgrade).
cat > "${INSTALL_DIR}/openshift/99_worker-osimagestream.yaml" <<EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
labels:
machineconfiguration.openshift.io/mco-built-in: ""
pools.operator.machineconfiguration.openshift.io/worker: ""
name: worker
spec:
machineConfigSelector:
matchLabels:
machineconfiguration.openshift.io/role: worker
nodeSelector:
matchLabels:
node-role.kubernetes.io/worker: ""
osImageStream:
name: ${WORKER_COREOS_STREAM}
EOF
Comment on lines +261 to +295

@coderabbitai coderabbitai Bot May 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Validate and escape WORKER_COREOS_STREAM before using it in sed/YAML.

WORKER_COREOS_STREAM is used unescaped in replacements and heredoc output; unexpected characters can break manifest generation or patching.

Suggested hardening 
 if [[ -n "${WORKER_COREOS_STREAM:-}" ]]; then
+    if [[ ! "${WORKER_COREOS_STREAM}" =~ ^[a-zA-Z0-9._-]+$ ]]; then
+        echo "[ERROR] Invalid WORKER_COREOS_STREAM: ${WORKER_COREOS_STREAM}"
+        exit 1
+    fi
+    esc_stream="$(printf '%s' "${WORKER_COREOS_STREAM}" | sed 's/[\/&]/\\&/g')"
     for bmh_file in "${INSTALL_DIR}"/openshift/99_openshift-cluster-api_hosts-*.yaml; do
         if ! grep -q 'installer.openshift.io/role: control-plane' "${bmh_file}"; then
-            sed -i "s/coreos.openshift.io\/stream: .*/coreos.openshift.io\/stream: ${WORKER_COREOS_STREAM}/" "${bmh_file}"
+            sed -i "s/coreos.openshift.io\/stream: .*/coreos.openshift.io\/stream: ${esc_stream}/" "${bmh_file}"
         fi
     done
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if [[ -n "${WORKER_COREOS_STREAM:-}" ]]; then
for bmh_file in "${INSTALL_DIR}"/openshift/99_openshift-cluster-api_hosts-*.yaml; do
if ! grep -q 'installer.openshift.io/role: control-plane' "${bmh_file}"; then
sed -i "s/coreos.openshift.io\/stream: .*/coreos.openshift.io\/stream: ${WORKER_COREOS_STREAM}/" "${bmh_file}"
fi
done
# Patch worker MachineSet hostSelector to match the new stream
for ms_file in "${INSTALL_DIR}"/openshift/99_openshift-cluster-api_worker-machineset-*.yaml; do
sed -i "s/coreos.openshift.io\/stream: .*/coreos.openshift.io\/stream: ${WORKER_COREOS_STREAM}/" "${ms_file}"
done
# Set the worker MachineConfigPool to use the specified stream for
# the on-disk OS. Requires the OSStreams feature gate (TechPreviewNoUpgrade).
cat > "${INSTALL_DIR}/openshift/99_worker-osimagestream.yaml" <<EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
labels:
machineconfiguration.openshift.io/mco-built-in: ""
pools.operator.machineconfiguration.openshift.io/worker: ""
name: worker
spec:
machineConfigSelector:
matchLabels:
machineconfiguration.openshift.io/role: worker
nodeSelector:
matchLabels:
node-role.kubernetes.io/worker: ""
osImageStream:
name: ${WORKER_COREOS_STREAM}
EOF
if [[ -n "${WORKER_COREOS_STREAM:-}" ]]; then
if [[ ! "${WORKER_COREOS_STREAM}" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "[ERROR] Invalid WORKER_COREOS_STREAM: ${WORKER_COREOS_STREAM}"
exit 1
fi
esc_stream="$(printf '%s' "${WORKER_COREOS_STREAM}" | sed 's/[\/&]/\\&/g')"
for bmh_file in "${INSTALL_DIR}"/openshift/99_openshift-cluster-api_hosts-*.yaml; do
if ! grep -q 'installer.openshift.io/role: control-plane' "${bmh_file}"; then
sed -i "s/coreos.openshift.io\/stream: .*/coreos.openshift.io\/stream: ${esc_stream}/" "${bmh_file}"
fi
done
# Patch worker MachineSet hostSelector to match the new stream
for ms_file in "${INSTALL_DIR}"/openshift/99_openshift-cluster-api_worker-machineset-*.yaml; do
sed -i "s/coreos.openshift.io\/stream: .*/coreos.openshift.io\/stream: ${esc_stream}/" "${ms_file}"
done
# Set the worker MachineConfigPool to use the specified stream for
# the on-disk OS. Requires the OSStreams feature gate (TechPreviewNoUpgrade).
cat > "${INSTALL_DIR}/openshift/99_worker-osimagestream.yaml" <<EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
labels:
machineconfiguration.openshift.io/mco-built-in: ""
pools.operator.machineconfiguration.openshift.io/worker: ""
name: worker
spec:
machineConfigSelector:
matchLabels:
machineconfiguration.openshift.io/role: worker
nodeSelector:
matchLabels:
node-role.kubernetes.io/worker: ""
osImageStream:
name: ${WORKER_COREOS_STREAM}
EOF
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/baremetal/lab/ipi/install/baremetal-lab-ipi-install-commands.sh`
around lines 261 - 290, The code uses WORKER_COREOS_STREAM directly in sed
replacements and an unquoted heredoc which can break manifests if the value
contains unexpected characters; add validation that WORKER_COREOS_STREAM matches
a safe pattern (e.g. ^[A-Za-z0-9._-]+$) and fail early if it doesn't, then use
the validated value (e.g. WORKER_COREOS_STREAM_SANITIZED) in the sed commands
and the heredoc for 99_worker-osimagestream.yaml; also switch sed to a delimiter
that reduces escaping issues (for example use |) and ensure all replacements
reference the sanitized variable (occurrences in the for loops and the heredoc
that create MachineConfigPool).

fi

### Inject customized manifests
echo -e "\n[INFO] The following manifests will be included at installation time:"
find "${SHARED_DIR}" \( -name "manifest_*.yml" -o -name "manifest_*.yaml" \)
Expand Down
2 changes: 2 additions & 0 deletions ci-operator/step-registry/baremetal/lab/ipi/install/baremetal-lab-ipi-install-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ ref:
default: "0"
- name: ADDITIONAL_WORKER_ARCHITECTURE
default: ""
- name: WORKER_COREOS_STREAM
default: ""
- name: ADDITIONAL_WORKERS_DAY2
default: "true"
- name: AUX_HOST
Expand Down