OCPCLOUD-3513: Verify manifests in e2e

[hongkailiu]

/hold

It requires openshift/release#80470 to get in first.

Summary by CodeRabbit

• Tests
  • Enhanced end-to-end testing pipeline with new manifest verification step
  • Added validation to ensure specified Custom Resource Definitions are not present during testing

[openshift-ci-robot]

@hongkailiu: This pull request references OCPCLOUD-3513 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

/hold

It requires openshift/release#80470 to get in first.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Walkthrough

This PR enhances the E2E test pipeline by adding a manifest verification gate. A new script validates that the cluster manifests are accessible and that a specific set of CRDs do not exist, then the E2E test orchestration is updated to invoke this verification before running the main test process.

Changes

E2E Manifest Verification

Layer / File(s)	Summary
Manifest and CRD verification script openshift/e2e-verify-manifests.sh	New script enforces strict error handling, fetches manifest YAML via oc get -f, and validates that declared CRDs are absent by iterating through a list, querying each with oc get crd --ignore-not-found -o name, and failing with exit code 1 if any are found.
E2E test orchestration openshift/e2e-tests.sh	Script computes its location (SCRIPT_DIR) and executes the adjacent verification script before proceeding with temp directory creation, cluster-capi-operator clone, and make e2e.

Sequence Diagram

sequenceDiagram
  participant e2e-tests.sh
  participant e2e-verify-manifests.sh
  participant Cluster
  e2e-tests.sh->>e2e-verify-manifests.sh: invoke
  e2e-verify-manifests.sh->>Cluster: oc get -f (fetch manifest)
  Cluster-->>e2e-verify-manifests.sh: manifest YAML
  e2e-verify-manifests.sh->>Cluster: oc get crd (verify CRDs absent)
  Cluster-->>e2e-verify-manifests.sh: --ignore-not-found
  alt any CRD found
    e2e-verify-manifests.sh->>e2e-verify-manifests.sh: exit 1
  else all CRDs absent
    e2e-verify-manifests.sh-->>e2e-tests.sh: continue
    e2e-tests.sh->>e2e-tests.sh: make e2e
  end

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Ipv6 And Disconnected Network Test Compatibility	❓ Inconclusive	Custom check execution failed before a final verdict was produced.	Retry the review run. If this persists, inspect pre-merge custom-check logs for infrastructure or agent runtime failures.

✅ Passed checks (14 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	PR changes only add/modify openshift/e2e-tests.sh and e2e-verify-manifests.sh; neither contains Ginkgo It/Describe/Context/When test titles with dynamic values.
Test Structure And Quality	✅ Passed	PR #615 changes only openshift/*.sh scripts (e2e-tests.sh, e2e-verify-manifests.sh); there are no Ginkgo tests/It blocks to evaluate against the requested quality rules.
Microshift Test Compatibility	✅ Passed	git diff --name-only origin/main..HEAD shows only openshift/e2e-tests.sh and openshift/e2e-verify-manifests.sh changed; no new/modified Ginkgo Go e2e tests to assess for MicroShift APIs.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	PR changes only Bash e2e helper scripts (e2e-tests.sh, e2e-verify-manifests.sh); no new/modified Ginkgo (_test.go) SNO assumptions to review.
Topology-Aware Scheduling Compatibility	✅ Passed	PR 615 only adds/updates e2e shell scripts: e2e-tests.sh runs e2e-verify-manifests.sh, which just checks for unexpected CRDs via oc get; no operator/controller or scheduling constraints are changed.
Ote Binary Stdout Contract	✅ Passed	The PR adds shell scripts (bash), not Go code. OTE Binary Stdout Contract applies only to process-level Go code (main, init, TestMain, BeforeSuite, etc.), not external CI/CD scripts.
No-Weak-Crypto	✅ Passed	Scanned openshift/e2e-tests.sh and e2e-verify-manifests.sh; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB/openssl usage and no secret/token comparison logic found.
Container-Privileges	✅ Passed	PR #615 only modifies openshift/e2e-tests.sh and openshift/e2e-verify-manifests.sh; both contain no privileged/hostPID/hostNetwork/hostIPC/SYS_ADMIN/allowPrivilegeEscalation settings.
No-Sensitive-Data-In-Logs	✅ Passed	Both scripts only echo generic progress messages and CRD identifiers; no password/token/API key/session/host/user data is logged, and there is no set -x/env dumping.
Title check	✅ Passed	The title clearly and specifically describes the main change: adding manifest verification to the e2e test process. It directly relates to the changeset which adds e2e-verify-manifests.sh and integrates it into e2e-tests.sh.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nrb for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openshift/e2e-verify-manifests.sh`:
- Around line 10-15: The loop that currently uses "if oc get crd ${crd}; then"
can mask API/auth/network failures as CRD absence; replace that conditional with
an explicit run of oc get crd "${crd}" capturing both its exit code and stderr
(e.g., store output="$(oc get crd "${crd}" 2>&1)"; status=$? ), then: if
status==0 -> treat as found and fail (same error/exit), else if output contains
"NotFound" or "not found" -> treat as absent and continue, else treat as an
API/auth/network error and exit non-zero with a clear error message. Apply this
change inside the for crd in "${arr[@]}"; do loop around the oc get crd ${crd}
invocation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7c396b68-ca00-40ae-a8a5-9d625cf83f39

📥 Commits

Reviewing files that changed from the base of the PR and between 77b3287 and 8041741.

📒 Files selected for processing (2)

• openshift/e2e-tests.sh
• openshift/e2e-verify-manifests.sh

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@docs/superpowers/specs/2026-06-11-exclude-awsmachinepools-openshift-design.md`:
- Around line 92-103: The patch for capa-manager-role fails to delete
awsmachinepools RBAC because the design combines three resources into one rule
without verbs so $patch: delete doesn't match the separately-generated
PolicyRules; update the patch for the ClusterRole named capa-manager-role to
include three distinct rules that exactly match the generated rules (one for
"awsmachinepools", one for "awsmachinepools/finalizers", one for
"awsmachinepools/status") including the same verbs arrays, each annotated with
$patch: delete so the kustomize patch matches and removes those PolicyRules.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3a7c3e13-b775-4061-a7f4-394d08149654

📥 Commits

Reviewing files that changed from the base of the PR and between 8041741 and cb3a1a4.

📒 Files selected for processing (4)

• docs/superpowers/plans/2026-06-11-exclude-awsmachinepools-openshift.md
• docs/superpowers/specs/2026-06-11-exclude-awsmachinepools-openshift-design.md
• openshift/e2e-tests.sh
• openshift/e2e-verify-manifests.sh

✅ Files skipped from review due to trivial changes (1)

• docs/superpowers/plans/2026-06-11-exclude-awsmachinepools-openshift.md

🚧 Files skipped from review as they are similar to previous changes (2)

• openshift/e2e-verify-manifests.sh
• openshift/e2e-tests.sh

[nader-ziada]

will you be able to get this in before the PR merges or is that for later? if later, can you create an issue for it if there isn't one

[hongkailiu]

This array with "pod.not-exist.io" and "svc.not-exist.io" will work after openshift/release#80470 gets in. They are the placeholder until we get the list in OCPCLOUD-3537.

The array is the list of CRDs that are not installed on the cluster and neither "pod.not-exist.io" nor "svc.not-exist.io" is. So we can merge this one as it is.

When https://redhat.atlassian.net/browse/OCPCLOUD-3537 is done, I am supposed to refresh the list with the real ones to filter out. I added an item into DoD to remind me of it.

[mdbooth]

Firstly, I don't think that oc is doing anything for us here. This is essentially a check that the CAPI installer works, as everything in the manifests should have been installed. We have other test for that. Lets keep this to a simple, offline test run in the verify step.

I think a more robust method, which would also be simpler to review, would be to have a file containing a specific allowed list. Perhaps a simple list of CRD names, one per line. You'd also need one of these files for each profile. The verify step could parse CRDs out of the target profile's manifests and ensure that the resulting set is identical to the expected set. This also makes changes to the expected set very easy to review.

Thoughts?

[hongkailiu]

/test e2e-aws-capi-techpreview

[openshift-ci]

@hongkailiu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-serial-1of2	a0cec9a	link	true	/test e2e-aws-serial-1of2
ci/prow/e2e-aws-serial-2of2	a0cec9a	link	true	/test e2e-aws-serial-2of2
ci/prow/e2e-aws-capi-techpreview	a0cec9a	link	true	/test e2e-aws-capi-techpreview

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.