Test Fix: flakiness fix (hopefully) for ocsp-stapling scripts#10049
Test Fix: flakiness fix (hopefully) for ocsp-stapling scripts#10049sebastian-carpenter wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Reduces flakiness in OCSP stapling test scripts by replacing fixed sleeps with readiness probes (port-open checks and ready-file creation/waits), including improved handling for UDP-ready-file creation.
Changes:
- Add
wait_for_port()polling to ensure OpenSSL OCSP responders have actually opened their listening ports before continuing. - Replace several fixed
sleepdelays withwait_for_readyFile(and add missing ready-file cleanup in a few DTLS cases). - Create the server “ready file” during UDP accept paths to support readiness signaling for DTLS runs.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
wolfssl/test.h |
Create ready-file during UDP accept path; fix typo in comment. |
scripts/ocsp-stapling_tls13multi.test |
Add port readiness probing for multiple OCSP responders; use ready-file wait instead of sleeps. |
scripts/ocsp-stapling2.test |
Add port readiness probing; increase OpenSSL server wait timeout; use ready-file wait instead of sleeps. |
scripts/ocsp-stapling.test |
Add port readiness probing; increase OpenSSL server wait timeout; remove sleeps in favor of ready-file waits. |
scripts/ocsp-stapling-with-wolfssl-responder.test |
Replace sleeps with wait_for_readyFile in more places. |
scripts/ocsp-stapling-with-ca-as-responder.test |
Add port readiness probing for OpenSSL OCSP responder startup. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Jenkins retest this please. |
|
Worth noting that this test failed the first time for ocsp responder and passed the second time. It failed because it was unable to bind a tcp port. Might be good to put a tighter retry loop on some areas of the test to address this. Specifically, the openssl server areas where ephemeral ports are not used. |
julek-wolfssl
left a comment
julek-wolfssl
left a comment
There was a problem hiding this comment.
If the flakiness is due to port race conditions, wouldn't it make more sense to bwrap the tests? With bwrap we could omit the rest of these changes.
julek-wolfssl
assigned
sebastian-carpenter and unassigned
douzzer,
wolfSSL-Bot and
julek-wolfssl
|
Closing in favor of #10177 |
Description
scripts/ocsp-stapling* tends to fail randomly. This seems like a failure to grab a port as well as the ready file not being written/checked.
Moved port testing function to right before a port is used. This will help prevent race conditions though it won't remove them.
Switched wolfSSL servers over to ephemeral ports. This doesn't really work well for the openssl server's but there is some stuff in place to help with this.
Added missing readyfile checks.
Added ready file creation when using udp.
Testing
Ran the ocsp-stapling tests. They seem to work fine but the issue itself is hard to reproduce. We will see if this fixes it.
Checklist