Skip to content

fix: pre-allocate open ports for rebalance integration test#639

Merged
enochtangg merged 1 commit into
mainfrom
fix/flakey-rebalance-integration-test
May 15, 2026
Merged

fix: pre-allocate open ports for rebalance integration test#639
enochtangg merged 1 commit into
mainfrom
fix/flakey-rebalance-integration-test

Conversation

@enochtangg

Copy link
Copy Markdown
Contributor

The flaky rebalance integration test occasionally fails on GHA due to ADDR_IN_USE on the gRPC port, but always passes consistently locally. Each of the 8 consumers in the test was assigned one gRPC port and restarted 16 times on that same port, with only a 1-second delay between restarts. The closed socket might be staying around longer than the delay and restarted taskbroker is trying to bind on the same port.

This PR pre-allocate one gRPC port per (consumer, restart) pair (128 total) at test start and each restart binds a fresh port that has never been used.

@enochtangg enochtangg requested a review from a team as a code owner May 15, 2026 17:56
@enochtangg enochtangg merged commit 157c0ee into main May 15, 2026
24 checks passed
@enochtangg enochtangg deleted the fix/flakey-rebalance-integration-test branch May 15, 2026 20:24
@sentry-taskbroker-fast-revert-bot

Copy link
Copy Markdown

PR reverted: 0071d96

sentry-taskbroker-fast-revert-bot Bot pushed a commit that referenced this pull request May 18, 2026
This reverts commit 157c0ee.

Co-authored-by: untitaker <837573+untitaker@users.noreply.github.com>
@untitaker untitaker mentioned this pull request May 18, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants