Skip to content

fix(e2e): serialize Kubernetes HA deployment tests - #3600

Closed
drew wants to merge 1 commit into
codex/fix-podman-e2e-workload-userfrom
codex/fix-kubernetes-ha-e2e-serialization
Closed

drew wants to merge 1 commit into
codex/fix-podman-e2e-workload-userfrom
codex/fix-kubernetes-ha-e2e-serialization

Conversation

@drew

@drew drew commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Serialize the Kubernetes HA E2E tests that mutate the same gateway deployment. Nextest launches each test in its own process, so the current in-process mutex cannot prevent one test from rolling the gateway while another is creating its sandbox.

Related Issue

Part of the #3583 stability campaign. The expanded Linux diagnostic run caught a gateway stream reset during the concurrent HA tests.

Changes

  • Restrict the HA test binary to one Nextest test thread in the Kubernetes E2E wrapper.
  • Document that deployment mutation tests require serial execution against a shared cluster.

Testing

  • bash -n e2e/rust/e2e-kubernetes.sh.
  • mise run pre-commit.
  • mise run ci.
  • Expanded GitHub Actions diagnostic for the combined stack tip.

Checklist

  • Conventional Commit with DCO sign-off.
  • Stacked with gh stack.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 23, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@drew
drew added this pull request to stack #3585 September 23, 2026 10:26
@drew drew added the automated-flake-find PRs produced by the automated E2E flake-finding campaign label Sep 23, 2026
@drew drew changed the title codex/fix kubernetes ha e2e serialization fix(e2e): serialize Kubernetes HA deployment tests Sep 23, 2026
@drew

drew commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator Author

Closing to narrow the automated flake investigation to the four selected fixes (#3583/#3588, #3598, #3604, #3608). The findings remain available here for later follow-up.

@drew drew closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated-flake-find PRs produced by the automated E2E flake-finding campaign

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant