Skip to content

fix: increase wait() timeout for multi-message consumer tests#1551

Draft
B-Whitt wants to merge 1 commit into
ansible:mainfrom
B-Whitt:worktree-fix+flaky-wait-timeout
Draft

fix: increase wait() timeout for multi-message consumer tests#1551
B-Whitt wants to merge 1 commit into
ansible:mainfrom
B-Whitt:worktree-fix+flaky-wait-timeout

Conversation

@B-Whitt

@B-Whitt B-Whitt commented May 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Tests that send multiple websocket messages before wait() can flake under CI load — the default 1-second timeout cancels the consumer task before all messages are processed
  • test_multiple_rules_for_one_event was observed failing with assert 1 == 2 because only one of two actions was processed within the 1s window
  • Added explicit timeout=TIMEOUT (5s) to the 5 multi-message tests only, matching the pattern already used for receive_json_from(timeout=TIMEOUT) in the same file
  • Single-message tests keep the default timeout for fast failure detection

Test plan

  • test_multiple_rules_for_one_event passes locally
  • All 33 tests in test_consumer.py pass (72s)
  • Changed file passes flake8 cleanly
  • CI passes on this PR

🤖 Assisted by Claude Opus

@B-Whitt B-Whitt requested a review from a team as a code owner May 5, 2026 13:12
@coderabbitai

coderabbitai Bot commented May 5, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@B-Whitt has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 45 minutes and 48 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: ce12c511-34c0-4915-9d93-6cddfdbcbab0

📥 Commits

Reviewing files that changed from the base of the PR and between 9c591d8 and 1b79d75.

📒 Files selected for processing (1)
  • tests/integration/wsapi/test_consumer.py
📝 Walkthrough

Walkthrough

Integration tests in the WebSocket consumer module add timeout constraints to synchronization waits by replacing bare wait() calls with wait(timeout=TIMEOUT) across 13 test functions, preventing indefinite hangs.

Changes

WebSocket Wait Timeout Additions

Layer / File(s) Summary
Test Wait Synchronization
tests/integration/wsapi/test_consumer.py
Thirteen test functions (test_handle_workers_with_validation_errors, test_handle_jobs, test_handle_events, test_handle_actions_multiple_firing, test_handle_actions_with_empty_job_uuid, test_handle_actions, test_rule_status_with_multiple_failed_actions, test_handle_heartbeat, test_handle_heartbeat_running_status, test_multiple_rules_for_one_event, test_controller_job_url, test_receive_object_not_exist, test_insert_audit_rule_invalid_activation) update their ws_communicator.wait() and communicator.wait() calls to use wait(timeout=TIMEOUT), applying a shared timeout constant across all websocket synchronization points.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and specifically describes the main change: adding explicit timeouts to wait() calls in multi-message consumer tests to fix flakiness.
Description check ✅ Passed The PR description clearly explains the problem (flaky tests due to short timeout), the solution (explicit 5s timeout), and testing performed.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov-commenter

codecov-commenter commented May 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.99%. Comparing base (9aba91d) to head (1b79d75).

@@           Coverage Diff           @@
##             main    #1551   +/-   ##
=======================================
  Coverage   91.99%   91.99%           
=======================================
  Files         241      241           
  Lines       10958    10958           
=======================================
  Hits        10081    10081           
  Misses        877      877           
Flag Coverage Δ
unit-int-tests-3.11 91.99% <ø> (ø)
unit-int-tests-3.12 91.99% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@B-Whitt B-Whitt force-pushed the worktree-fix+flaky-wait-timeout branch from 9c591d8 to 552c3bd Compare May 5, 2026 13:29
@B-Whitt B-Whitt changed the title fix: use explicit timeout for ws_communicator.wait() in consumer tests fix: increase wait() timeout for multi-message consumer tests May 5, 2026
@B-Whitt

B-Whitt commented May 5, 2026

Copy link
Copy Markdown
Contributor Author

/run-e2e

@jcraiglo1 jcraiglo1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@jcraiglo1 jcraiglo1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice and elegant

Tests that send multiple websocket messages before calling wait()
can flake under CI load because the default 1-second timeout cancels
the consumer task before all messages are processed. Add explicit
timeout=TIMEOUT (5s) to the 5 multi-message tests only, matching
the pattern already used for receive_json_from() in the same file.
Single-message tests keep the default for fast failure detection.

Assisted-by: Claude Opus
@B-Whitt B-Whitt force-pushed the worktree-fix+flaky-wait-timeout branch from 552c3bd to 1b79d75 Compare May 5, 2026 18:42
@sonarqubecloud

sonarqubecloud Bot commented May 5, 2026

Copy link
Copy Markdown

@B-Whitt B-Whitt marked this pull request as draft May 12, 2026 17:52
@ptoscano

Copy link
Copy Markdown
Contributor

@B-Whitt would you please link logs of jobs that fail, and would be fixed by this? nothing potentially wrong with this, I simply want to check what is the issue supposed to be fixed

thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants