man: document that send/recv can see -EAGAIN without MSG_DONTWAIT#1601
man: document that send/recv can see -EAGAIN without MSG_DONTWAIT#1601tchaikov wants to merge 1 commit into
Conversation
The send/recv prep man pages imply -EAGAIN is only expected when the application sets MSG_DONTWAIT. In practice io_uring can also complete a request that did not set MSG_DONTWAIT with -EAGAIN: when it cannot make progress, it stops retrying rather than waiting indefinitely (see kernel commit c16bda37594f "io_uring/poll: allow some retries for poll triggering spuriously", which bounds the retry attempts). Applications submitting blocking-style socket sends/recvs (no MSG_DONTWAIT) can therefore observe -EAGAIN under load and must treat it as transient and reissue. Note this in the ERRORS section of the send, sendmsg, recv and recvmsg prep pages. Signed-off-by: Kefu Chai <k.chai@proxmox.com>
|
This is essentially a "will never happen in an app" case, but added so that malicious/odd reproducers don't get stuck forever. Curious if this is something you actually ran into, or whether you just skimmed the source and spotted this? |
The io_uring backend can complete a socket send with -EAGAIN and leak it to the caller: when arming a poll fails the request goes to io-wq, which does not retry EAGAIN for SOCK_NONBLOCK sockets. A send can complete with -EAGAIN even without MSG_DONTWAIT; this is expected io_uring behaviour and must be retried (documentation requested in axboe/liburing#1601). Intercept the -EAGAIN completion in complete_with() and re-issue through the poll-based do_sendmsg()/do_send(), like the aio/epoll backends. The handling lives in the shared completion base classes so both io_uring backends recover from it. Observed under load on riscv64 (io_uring is the auto-selected backend): unittest-seastar-socket's test_preemptive_down() aborts with EAGAIN. Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
|
@axboe hi Jens, it's real, not from skimming the source. actually, it showed up when testing Ceph's crimson test suite, which runs on Seastar's io_uring reactor backend, on a riscv64 box. the socket test it only shows up under heavy CPU contention. the test runs Seastar on 2 cores ( for i in $(seq 12); do
taskset -c 0-1 yes >/dev/null &
done
taskset -c 0-1 ./unittest-seastar-socket --reactor-backend io_uring --smp 2With the cores that contended, 2 of 3 runs abort. ftrace on the io_uring events caught the path: a partial send that can't make further progress while the reactor is starved, hitting the retry cap from kernel commit c16bda37594f. So yes, pathological scheduling, but a real workload rather than a crafted reproducer. we fixed it in Seastar by treating |
The send/recv prep man pages imply -EAGAIN is only expected when the application sets MSG_DONTWAIT. In practice io_uring can also complete a request that did not set MSG_DONTWAIT with -EAGAIN: when it cannot make progress, it stops retrying rather than waiting indefinitely (see kernel commit c16bda37594f "io_uring/poll: allow some retries for poll triggering spuriously", which bounds the retry attempts).
Applications submitting blocking-style socket sends/recvs (no MSG_DONTWAIT) can therefore observe -EAGAIN under load and must treat it as transient and reissue. Note this in the ERRORS section of the send, sendmsg, recv and recvmsg prep pages.
git request-pull output:
Click to show/hide pull request guidelines
Pull Request Guidelines
notification, use
[GIT PULL]as a prefix in your PR title.Commit message format rules:
Signed-off-bytag with your real name and email. For example:The description should be word-wrapped at 72 chars. Some things should
not be word-wrapped. They may be some kind of quoted text - long
compiler error messages, oops reports, Link, etc. (things that have a
certain specific format).
Note that all of this goes in the commit message, not in the pull
request text. The pull request text should introduce what this pull
request does, and each commit message should explain the rationale for
why that particular change was made. The git tree is canonical source
of truth, not github.
Each patch should do one thing, and one thing only. If you find yourself
writing an explanation for why a patch is fixing multiple issues, that's
a good indication that the change should be split into separate patches.
If the commit is a fix for an issue, add a
Fixestag with the issueURL.
Don't use GitHub anonymous email like this as the commit author:
Use a real email address!
Commit message example:
By submitting this pull request, I acknowledge that: