Skip to content

fix(cli): retry sync during supervisor reconnect - #3602

Closed
drew wants to merge 1 commit into
codex/fix-e2e-gateway-port-collisionfrom
codex/fix-sync-retry-readiness-rollout
Closed

drew wants to merge 1 commit into
codex/fix-e2e-gateway-port-collisionfrom
codex/fix-sync-retry-readiness-rollout

Conversation

@drew

@drew drew commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Keep a file sync retry alive through a brief sandbox-not-ready response after a gateway pod roll interrupts its SSH stream. The Kubernetes HA diagnostic showed a download that retried a broken stream, then failed while the supervisor was reconnecting.

Related Issue

Part of the #3583 stability campaign. This is a localized bug exposed by the expanded HA E2E run.

Changes

  • Allow bounded sync retries for sandbox-not-ready only after a retryable transport failure.
  • Increase the bounded attempt count to cover supervisor reconnect during a gateway roll.
  • Add retry classification and observed-sequence unit tests.
  • Document the HA sync behavior in the architecture and sandbox user guide.

Testing

  • Focused CLI retry filter tests: 4 passed.
  • Observed transport/readiness retry sequence test: passed.
  • mise run pre-commit.
  • mise run ci.
  • Expanded GitHub Actions diagnostic on this PR's stack tip.

Checklist

  • Conventional Commit with DCO sign-off.
  • Stacked with gh stack.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 23, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@drew
drew added this pull request to stack #3585 September 23, 2026 10:51
@drew drew changed the title codex/fix sync retry readiness rollout fix(cli): retry sync during supervisor reconnect Sep 23, 2026
@drew drew added the automated-flake-find PRs produced by the automated E2E flake-finding campaign label Sep 23, 2026
@github-actions

Copy link
Copy Markdown

@drew drew mentioned this pull request Sep 23, 2026
4 tasks
@drew

drew commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator Author

Closing to narrow the automated flake investigation to the four selected fixes (#3583/#3588, #3598, #3604, #3608). The findings remain available here for later follow-up.

@drew drew closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated-flake-find PRs produced by the automated E2E flake-finding campaign

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant