diff --git a/doc/as_sender.md b/doc/as_sender.md index 98417d50f..d5e4be239 100644 --- a/doc/as_sender.md +++ b/doc/as_sender.md @@ -54,7 +54,8 @@ struct frame_cb { Three pointers. Twenty-four bytes on a 64-bit platform. The `resume` pointer holds the sender's completion callback. The `destroy` pointer -is a no-op - the sender owns its own lifetime. The `data` pointer +is reached only when the awaitable's segment is given up: it destroys +the awaitable and completes the receiver stopped. The `data` pointer points back to the operation state. `std::coroutine_handle<>::from_address(&cb)` produces a handle that, @@ -83,7 +84,7 @@ Here is what happens, step by step: inline. 3. Otherwise, the `frame_cb` is filled in: `resume` points to the - completion callback, `destroy` is a no-op, `data` points to the + completion callback, `destroy` gives the operation up, `data` points to the operation state. A `coroutine_handle<>` is manufactured from the `frame_cb`'s address. `await_suspend(handle, &env)` is called on the awaitable. diff --git a/doc/modules/ROOT/nav.adoc b/doc/modules/ROOT/nav.adoc index 77d026961..36ddca04f 100644 --- a/doc/modules/ROOT/nav.adoc +++ b/doc/modules/ROOT/nav.adoc @@ -11,6 +11,7 @@ ** xref:4.coroutines/4g.composition.adoc[Concurrent Composition] ** xref:4.coroutines/4h.allocators.adoc[Frame Allocators] ** xref:4.coroutines/4i.lambda-captures.adoc[Lambda Coroutine Captures] +** xref:4.coroutines/4j.lifetimes.adoc[Lifetime and Ownership] * xref:5.buffers/5.intro.adoc[Buffer Sequences] ** xref:5.buffers/5a.contiguous-buffers.adoc[Contiguous Buffers] ** xref:5.buffers/5b.composite-buffers.adoc[Composite Buffers] diff --git a/doc/modules/ROOT/pages/4.coroutines/4f.cancellation.adoc b/doc/modules/ROOT/pages/4.coroutines/4f.cancellation.adoc index 42d68dc75..798800793 100644 --- a/doc/modules/ROOT/pages/4.coroutines/4f.cancellation.adoc +++ b/doc/modules/ROOT/pages/4.coroutines/4f.cancellation.adoc @@ -157,7 +157,7 @@ include::example$snippets/4f_cancellation.cpp[tag=access_stop_token,indent=0] `std::coroutine_handle::destroy()` is the {cpp}20 primitive that frees a coroutine frame. It is not a cancellation mechanism. It has the same flaw as forceful thread interruption. The coroutine is torn down with no opportunity to complete pending I/O, release locks, or run RAII destructors in the expected order. -Capy exposes cpp:task::handle[handle]`()` and cpp:quitter::handle[handle]`()` so that Capy's own launchers (cpp:run_async[], cpp:run[]) and custom integrations can dispatch coroutines through executors. Calling `destroy()` on such a handle while the coroutine is being awaited by a parent produces undefined behavior. The destruction cascades back through the parent's continuation, re-entering frame destruction that is already in progress. +Capy exposes cpp:task::handle[handle]`()` and cpp:quitter::handle[handle]`()` so that Capy's own launchers (cpp:run_async[], cpp:run[]) and custom integrations can dispatch coroutines through executors. Calling `destroy()` on such a handle while the coroutine is being awaited by a parent produces undefined behavior: the parent still holds the frame and will destroy it again. A holder that must give up a suspended coroutine for good, such as an execution context being torn down, passes its continuation to cpp:abandon[] instead, which unwinds the chain from the top on its owner's executor. The rule: diff --git a/doc/modules/ROOT/pages/4.coroutines/4h.allocators.adoc b/doc/modules/ROOT/pages/4.coroutines/4h.allocators.adoc index 71d3c6339..cc7b17257 100644 --- a/doc/modules/ROOT/pages/4.coroutines/4h.allocators.adoc +++ b/doc/modules/ROOT/pages/4.coroutines/4h.allocators.adoc @@ -112,6 +112,13 @@ When starting many short-lived tasks together, a monotonic buffer resource can b include::example$snippets/4h_allocators.cpp[tag=batch_allocator] ---- +The resource must outlive every task started with it, including the unwind of +any that are abandoned. This function returns only after its tasks have +finished when the executor runs each one to completion before `run_async` +returns. With an executor that queues the work, keep the resource alive until +the tasks have finished. See +xref:4.coroutines/4j.lifetimes.adoc#_shutdown_ordering[Shutdown Ordering]. + === Scope Variables to Reduce Frame Size Clang decides frame layout after optimization, so declaration scope controls what it keeps. A variable whose scope spans a `co_await` stays in the frame until that scope ends. diff --git a/doc/modules/ROOT/pages/4.coroutines/4j.lifetimes.adoc b/doc/modules/ROOT/pages/4.coroutines/4j.lifetimes.adoc new file mode 100644 index 000000000..f2b5dcddd --- /dev/null +++ b/doc/modules/ROOT/pages/4.coroutines/4j.lifetimes.adoc @@ -0,0 +1,233 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + += Lifetime and Ownership +:page-mode: explanation + +A coroutine outlives the call that created it. It can suspend in one function +and resume long after that function has returned, on another thread, after the +objects it was given have gone. Most coroutine bugs are lifetime bugs: a +reference that outlived what it referred to, or a frame that outlived what it +held. + +This page gathers the rules for composing Capy coroutines: what a coroutine +frame keeps alive, which arguments are safe to pass by reference, who owns a +suspended coroutine, and what happens to it when the thing it waits on is torn +down. + +== What a Frame Keeps Alive + +Calling a coroutine function allocates a _frame_ and copies or moves each +parameter into it. Locals declared in the body live in the frame too. The +frame lives until the coroutine finishes, or until it is destroyed while +suspended. + +What the frame copies is the parameter, not what the parameter refers to. A +`std::string` parameter is a string the frame owns. A `std::string const&` +parameter is a reference the frame holds to a string someone else owns, and +nothing keeps that string alive. + +A cpp:task[] is lazy: the coroutine does not start until it is awaited or +launched. The `task` object owns the frame. Awaiting a task lends the frame to +the awaiting coroutine, which resumes when it finishes; destroying an unstarted +or finished task destroys its frame. + +== Arguments by Reference + +A reference parameter is safe when the referenced object outlives the +coroutine. The common safe case is a task awaited in the same full expression +that creates its argument: + +[source,cpp] +---- +include::example$snippets/4j_lifetimes.cpp[tag=ref_param_awaited] +---- + +The same function is unsafe as soon as the task is stored and awaited later: + +[source,cpp] +---- +include::example$snippets/4j_lifetimes.cpp[tag=ref_param_deferred] +---- +<1> The temporary string is destroyed at the semicolon. The task is lazy, so + its body has not run yet. +<2> The body runs here and reads the destroyed string. + +The fix is to take by value whatever a task reads after it may have been +stored. The string is then moved into the frame: + +[source,cpp] +---- +include::example$snippets/4j_lifetimes.cpp[tag=value_param] +---- + +Tasks are stored more often than it looks. Every launch stores one: +cpp:run_async[] starts the task through its executor, which can run it after +the full expression that launched it has ended: + +[source,cpp] +---- +include::example$snippets/4j_lifetimes.cpp[tag=run_async_temporary,indent=0] +---- +<1> Undefined behavior: the string may be gone before `count_bytes` reads it. + +[source,cpp] +---- +include::example$snippets/4j_lifetimes.cpp[tag=run_async_owned,indent=0] +---- + +Collecting tasks in a range and awaiting them together with cpp:when_all[] or +cpp:when_any[] stores them too. See the note under +xref:4.coroutines/4g.composition.adoc#_fan_outfan_in[Fan-Out/Fan-In]. + +The rules: + +* Take parameters by value unless every caller awaits the task at once. +* A reference parameter is fine for a task that is always awaited in the + expression that creates it, such as a helper a coroutine calls and + immediately awaits. +* Anything passed to a launched or stored task must be owned by its frame or + outlive it. + +== Lambda Captures + +A lambda coroutine's captures live in the lambda object, not in the frame. A +lambda invoked and discarded in one expression is gone by the time the +coroutine resumes, and its captures with it. This applies to captures by value +as much as by reference, and to `this`. + +Pass what the coroutine needs as parameters instead, which the frame copies. +xref:4.coroutines/4i.lambda-captures.adoc[Lambda Coroutine Captures] shows the +failure and the safe patterns. + +== Buffers + +An I/O operation does not copy the memory a buffer refers to. The storage must +outlive the operation, which it does when the operation is awaited from the +frame that owns the storage. Returning or storing the awaitable instead of +awaiting it lets the storage go first. +xref:5.buffers/5a.contiguous-buffers.adoc#_lifetime_management[Lifetime Management] +shows both cases. + +Take a buffer sequence by value in a coroutine that stores it, for the same +reason as any other parameter. The buffer descriptors are copied; the memory +they describe is not. + +== Inside an Awaitable + +An awaitable is given an cpp:io_env[] pointer in `await_suspend`. The +environment belongs to the launcher that started the chain and lives until the +awaitable's coroutine resumes or is unwound, so the awaitable can keep the +pointer. The cpp:continuation[] it hands to an executor must keep a stable +address until the executor runs it, so it belongs in the awaitable, not on the +stack. + +xref:4.coroutines/4e.io-awaitable.adoc#_implementing_custom_ioawaitables[Implementing Custom IoAwaitables] +lists these rules, and the rules for stop callbacks. + +== Who Owns a Suspended Coroutine + +A chain of coroutines awaiting one another is split into _segments_. A segment +is the run of coroutines started together: the task cpp:run_async[] launches +and everything it awaits, a task cpp:run[] starts on another executor, or one +branch of cpp:when_all[] or cpp:when_any[]. The coroutine that started the +segment is its _owner_. It is suspended above the segment until the +segment's top task finishes. + +While the chain is suspended, whatever it waits on holds its continuation: an +executor's queue, a strand, a mutex, a pending I/O operation. Normally the +holder resumes it. A holder that will never resume it, because it is being torn +down, passes the continuation to cpp:abandon[] instead. The segment's owner +then destroys the segment from its top frame down, on its own executor, and +passes the unwind to the segment above. Nothing in an abandoned chain is +resumed: no code after a suspension point runs, no result is delivered, and +cpp:run_async[] calls neither of its handlers. + +What does run is destructors. Each frame's locals are destroyed in the usual +order, so RAII still releases what the chain held: + +[source,cpp] +---- +include::example$snippets/4j_lifetimes.cpp[tag=unwind] +---- +<1> Runs whether the coroutine finishes or is unwound while suspended. +<2> Not reached if the chain is abandoned while waiting: the frame is destroyed + without being resumed. + +Abandoning is not cancellation. To stop work, request a stop on the token the +work observes: it resumes with an error and returns through its own code. +Abandoning is what teardown does to work that was not stopped first. See +xref:4.coroutines/4f.cancellation.adoc[Stop Tokens and Cancellation]. + +Never call `destroy()` on the handle of a coroutine that another coroutine is +awaiting. The awaiting frame still holds it and destroys it again. A task whose +frame is destroyed while it is awaited, outside the unwind of an abandoned +chain, calls `std::terminate`. + +== Shutdown Ordering + +Destroying an execution context, such as a cpp:thread_pool[], abandons the +coroutines still queued or suspended in it, including those queued on its +strands. Each chain is unwound +on the executor of the coroutine that started it. When that executor belongs to +the context being destroyed, the unwind runs during the destruction. When it +belongs to another context, the unwind runs there later. + +Destroying an cpp:async_mutex[], cpp:async_event[] or cpp:async_waker[] is +different: each waiter is resumed with an error, as a stop request would +resume it, and returns through its own code. + +That gives the ordering rules: + +* Stop work and let it finish before destroying what it runs on, rather than + relying on teardown to unwind it. +* A chain whose frames hold objects of one context, or wait on one context's + operations, must finish before that context is destroyed. If it was started + on another context, its unwind would run after the first context is gone. +* The executors, memory resources and allocators given to cpp:run_async[] and + cpp:run[] must outlive the chains they start, including their unwind. +* No thread may post to a context during or after its destruction. + +The pattern is to request a stop, wait for the work to end, then destroy: + +[source,cpp] +---- +include::example$snippets/4j_lifetimes.cpp[tag=shutdown_order,indent=0] +---- +<1> The work observes the token and finishes through its own code. +<2> `join` waits for the outstanding work, then for the worker threads. +<3> The pool is destroyed with nothing left to abandon. + +== Summary + +[cols="1,2"] +|=== +| Situation | Rule + +| Task parameter +| By value, unless every caller awaits the task in the same expression. + +| Launched or stored task +| Everything it reads is owned by its frame or outlives it. + +| Lambda coroutine +| Pass what it needs as parameters, not captures. + +| Buffers +| The storage outlives the operation; await it, do not store it. + +| Awaitable +| Keep the `io_env` pointer; keep the `continuation` in the awaitable. + +| Suspended coroutine +| Its holder resumes it or abandons it; never `destroy()` an awaited handle. + +| Teardown +| Stop, wait for the work to end, then destroy the context. +|=== diff --git a/doc/modules/ROOT/pages/8.examples/8a.hello-task.adoc b/doc/modules/ROOT/pages/8.examples/8a.hello-task.adoc index 34b287488..a95304bc6 100644 --- a/doc/modules/ROOT/pages/8.examples/8a.hello-task.adoc +++ b/doc/modules/ROOT/pages/8.examples/8a.hello-task.adoc @@ -46,7 +46,7 @@ include::example$hello-task/hello_task.cpp[tag=pool,indent=0] cpp:thread_pool[] provides an execution context with worker threads. By default, it creates one thread per CPU core. -Call `pool.join()` before the pool is destroyed to wait for all outstanding work to finish. The destructor does *not* wait for queued work: `~thread_pool` calls `stop()`, which abandons any work that has not yet started, then joins the worker threads. Without the explicit `pool.join()`, `main` could destroy the pool and exit before `say_hello` ever runs, so `"Hello from Capy!"` might never print. +Call `pool.join()` before the pool is destroyed to wait for all outstanding work to finish. The destructor does *not* wait for queued work: `~thread_pool` calls `stop()` and joins the worker threads, then gives up any work that has not yet started. Without the explicit `pool.join()`, `main` could destroy the pool and exit before `say_hello` ever runs, so `"Hello from Capy!"` might never print. === Starting the Task diff --git a/doc/modules/ROOT/pages/8.examples/8n.custom-executor.adoc b/doc/modules/ROOT/pages/8.examples/8n.custom-executor.adoc index 089aebf19..719772f61 100644 --- a/doc/modules/ROOT/pages/8.examples/8n.custom-executor.adoc +++ b/doc/modules/ROOT/pages/8.examples/8n.custom-executor.adoc @@ -33,7 +33,7 @@ target_link_libraries(custom_executor PRIVATE Boost::capy) include::example$custom-executor/custom_executor.cpp[tag=inherit] ---- -The destructor must call `shutdown()` then `destroy()` to clean up coroutine state. +The destructor must call `shutdown()`, give up any work still queued with cpp:abandon[], then call `destroy()`. Destroying a service can post more work, and unwinding an abandoned chain can create a service, so the destructor repeats `destroy()` and the drain until a drain finds nothing. The queue holds copies of each cpp:continuation[], so the owner that cpp:abandon[] needs travels with the handle. === The Executor Concept @@ -57,7 +57,7 @@ include::example$custom-executor/custom_executor.cpp[tag=concept_check] include::example$custom-executor/custom_executor.cpp[tag=dispatch,indent=0] ---- -`dispatch` takes a `continuation&` and checks whether the caller is already running on the loop's thread. If so, it returns `c.h` directly for inline resumption via symmetric transfer. Otherwise it enqueues `c.h` and returns `noop_coroutine` so the caller continues without blocking. +`dispatch` takes a `continuation&` and checks whether the caller is already running on the loop's thread. If so, it returns `c.h` directly for inline resumption via symmetric transfer. Otherwise it enqueues a copy of `c` and returns `noop_coroutine` so the caller continues without blocking. `post` always enqueues, even if already on the right thread. diff --git a/doc/modules/ROOT/pages/8.examples/8q.gui-integration.adoc b/doc/modules/ROOT/pages/8.examples/8q.gui-integration.adoc index 0e966e020..932d10bfb 100644 --- a/doc/modules/ROOT/pages/8.examples/8q.gui-integration.adoc +++ b/doc/modules/ROOT/pages/8.examples/8q.gui-integration.adoc @@ -36,7 +36,7 @@ The program links no GUI library. It uses a stand-in instead. include::example$gui-integration/gui_integration.cpp[tag=gui_app] ---- -`gui_app` records the thread it is constructed on, holds a queue of coroutine handles, and exposes `post_to_gui_thread`. That last operation is the one primitive every toolkit already provides. Qt names it `QMetaObject::invokeMethod`, GTK `g_idle_add`, wxWidgets `wxEvtHandler::CallAfter`, Win32 `PostMessage`. +`gui_app` records the thread it is constructed on, holds a queue of continuations, and exposes `post_to_gui_thread`. That last operation is the one primitive every toolkit already provides. Qt names it `QMetaObject::invokeMethod`, GTK `g_idle_add`, wxWidgets `wxEvtHandler::CallAfter`, Win32 `PostMessage`. === The Event Loop @@ -158,7 +158,7 @@ The completion handler runs on the GUI thread, because the task completed there. The stand-in exists to prove the contract, not to model a toolkit. Five pieces change when the toolkit is real, and the rest does not: -* `post_to_gui_thread` forwards to the toolkit's own post operation, and the receiving side calls cpp:safe_resume[] on the handle. +* `post_to_gui_thread` forwards to the toolkit's own post operation, and the receiving side calls cpp:safe_resume[] on the handle. Work still queued when the loop is torn down is given up with cpp:abandon[], so its chain unwinds instead of leaking. * `gui_app::run` disappears. The toolkit already owns a loop, such as `QApplication::exec`, `g_main_loop_run`, or `wxApp::OnRun`. * `gui_app::quit` forwards to the toolkit's own quit, such as `QCoreApplication::quit`, `g_main_loop_quit`, or `wxAppConsole::ExitMainLoop`. * `label` becomes a real widget. Its thread check is the part the toolkit does not do for you. diff --git a/doc/modules/ROOT/pages/9.design/9i.TypeEraseAwaitable.adoc b/doc/modules/ROOT/pages/9.design/9i.TypeEraseAwaitable.adoc index 6ede79a11..8281f947a 100644 --- a/doc/modules/ROOT/pages/9.design/9i.TypeEraseAwaitable.adoc +++ b/doc/modules/ROOT/pages/9.design/9i.TypeEraseAwaitable.adoc @@ -30,14 +30,19 @@ struct vtable === Construct in await_ready (any_read_stream) -When there is no outer short-circuit, constructing in `await_ready` lets immediate completions skip `await_suspend` entirely: +When there is no outer short-circuit, constructing in `await_ready` lets immediate completions skip `await_suspend` entirely. A chain unwound while suspended never reaches `await_resume`, so the outer awaitable's destructor destroys the inner one; `owns_` marks the outer awaitable that constructed it, since another may be created on the same stream without being awaited: .Design sketch [source,cpp,role=pseudocode] ---- +~awaitable() { + if(owns_ && awaitable_active_) // unwound while suspended + vt_->destroy_awaitable(cached_awaitable_); +} + bool await_ready() { vt_->construct_awaitable(stream_, cached_awaitable_, buffers); - awaitable_active_ = true; + awaitable_active_ = owns_ = true; return vt_->await_ready(cached_awaitable_); // true → no suspend } @@ -48,36 +53,32 @@ std::coroutine_handle<> await_suspend(std::coroutine_handle<> h, io_env const* e io_result await_resume() { auto r = vt_->await_resume(cached_awaitable_); vt_->destroy_awaitable(cached_awaitable_); - awaitable_active_ = false; + awaitable_active_ = owns_ = false; return r; } ---- -=== Construct in await_suspend (any_write_stream) +=== Short-circuit before constructing (any_write_stream) -When the outer awaitable has a short-circuit (empty buffers), construction is deferred to `await_suspend` so the inner awaitable is never created on the fast path: +When the outer awaitable has a short-circuit (empty buffers), `await_ready` checks it first, so the inner awaitable is never created on the fast path. Otherwise construction proceeds as above: .Design sketch [source,cpp,role=pseudocode] ---- -bool await_ready() const noexcept { - return buffers_.empty(); // short-circuit, no construct -} - -std::coroutine_handle<> await_suspend(std::coroutine_handle<> h, io_env const* env) { +bool await_ready() { + if(buffers_.empty()) + return true; // short-circuit, no construct vt_->construct_awaitable(stream_, cached_awaitable_, buffers); - awaitable_active_ = true; - if(vt_->await_ready(cached_awaitable_)) - return h; // immediate → resume caller - return vt_->await_suspend(cached_awaitable_, h, env); + awaitable_active_ = owns_ = true; + return vt_->await_ready(cached_awaitable_); } io_result await_resume() { - if(!awaitable_active_) - return {std::error_code(), 0}; // short-circuited + if(!owns_) + return {std::error_code(), 0}; // short-circuited auto r = vt_->await_resume(cached_awaitable_); vt_->destroy_awaitable(cached_awaitable_); - awaitable_active_ = false; + awaitable_active_ = owns_ = false; return r; } ---- diff --git a/doc/modules/ROOT/pages/9.design/9k.Executor.adoc b/doc/modules/ROOT/pages/9.design/9k.Executor.adoc index f5e82fca7..7da9fb06c 100644 --- a/doc/modules/ROOT/pages/9.design/9k.Executor.adoc +++ b/doc/modules/ROOT/pages/9.design/9k.Executor.adoc @@ -126,7 +126,7 @@ Corosio confirms this in practice: its entire I/O layer -- sockets, acceptors, t == Why `continuation`, Not Raw `coroutine_handle<>` -The executor accepts cpp:continuation[]`&` rather than `std::coroutine_handle<>`. A cpp:continuation[] wraps the handle with a pointer-sized cpp:continuation::reserved[reserved] slot that the executor commandeers as its queue link, enabling zero-allocation queuing: +The executor accepts cpp:continuation[]`&` rather than `std::coroutine_handle<>`. A cpp:continuation[] wraps the handle with a pointer-sized cpp:continuation::reserved[reserved] slot that the executor commandeers as its queue link, enabling zero-allocation queuing, and the cpp:continuation::owner[owner] that cpp:abandon[] uses to unwind the chain when the coroutine is given up instead of resumed: [source,cpp] ---- @@ -174,12 +174,14 @@ These operations are public, not private with friendship. The reason is extensib === Executor Validity -An executor becomes invalid when its context's cpp:execution_context::shutdown[shutdown()] returns. After shutdown: +Once a context's cpp:execution_context::shutdown[shutdown()] has begun, it runs no more work. Its executors stay valid until the context is destroyed: -- cpp:Executor::dispatch[dispatch], cpp:Executor::post[post], cpp:Executor::on_work_started[on_work_started], cpp:Executor::on_work_finished[on_work_finished]: undefined behavior. -- Copy, comparison, cpp:Executor::context[context()]: valid until the context is destroyed. +- cpp:Executor::dispatch[dispatch] and cpp:Executor::post[post] never run the coroutine they are given. The executor passes the continuation to cpp:abandon[], at once or no later than the context's destruction, so the chain is unwound by its owner. +- cpp:Executor::on_work_started[on_work_started], cpp:Executor::on_work_finished[on_work_finished], copy, comparison and cpp:Executor::context[context()] remain valid. -This two-phase model exists because shutdown drains outstanding work. During the drain, executors must still be copyable (they are stored in pending operations) and comparable (for same-executor checks). Only the work-submission operations become invalid, because the context has stopped accepting new work. +Calling any member after the context is destroyed is undefined behavior. + +This model exists because teardown races with work that is still in flight. A coroutine finishing on another thread may post to a context that is shutting down; refusing the post would leave its chain suspended forever, while abandoning it unwinds the chain the same way as work that was already queued. == Why `context()` Returns `execution_context&` diff --git a/doc/strand-spec.md b/doc/strand-spec.md index 6b48d532e..b9e531fc3 100644 --- a/doc/strand-spec.md +++ b/doc/strand-spec.md @@ -230,9 +230,10 @@ strand_impl::~strand_impl() } ``` -`~strand_queue` (already implemented) destroys any pending wrappers -without resuming them. That covers the case where work was queued but -the inner executor never invoked the invoker before context teardown. +`strand_queue::abandon_batch` gives any pending continuations to +`abandon` without resuming them. That covers the case where work was +queued but the inner executor never invoked the invoker before context +teardown. ### Shutdown @@ -279,10 +280,9 @@ freed. service state, which may have been freed. 4. **Service shutdown with invoker queued but never invoked.** The - inner executor's destructor drops the queued continuation; the - coroutine handle is never destroyed; the promise's `shared_ptr` - never releases; impl and frame leak. Pre-existing behavior, not - introduced by this design. + inner context abandons the queued continuation, destroying the + invoker frame; its `shared_ptr` releases the impl, whose destructor + abandons what is still pending. 5. **Service shutdown with invoker mid-execution.** The invoker accesses the service only via the trailer in `operator delete` (cache-slot diff --git a/doc/unlisted/execution-executors.adoc b/doc/unlisted/execution-executors.adoc index 315ab1133..18078f94b 100644 --- a/doc/unlisted/execution-executors.adoc +++ b/doc/unlisted/execution-executors.adoc @@ -171,7 +171,9 @@ Executors have specific thread safety guarantees: == Executor Validity -An executor becomes invalid when its execution context shuts down: +An executor stays valid until its execution context is destroyed. Once +the context has begun shutting down it runs no more work: `dispatch` +and `post` give the coroutine to `abandon` instead of resuming it. [source,cpp] ---- @@ -182,8 +184,8 @@ auto ex = pool.get_executor(); // WARNING: Calling ex.dispatch() after pool destruction is undefined behavior ---- -The copy constructor and `context()` remain valid until the context is -destroyed, but work submission functions become undefined. +Calling any member after the context is destroyed is undefined +behavior. == When NOT to Use Executors Directly diff --git a/doc/unlisted/execution-thread-pool.adoc b/doc/unlisted/execution-thread-pool.adoc index 71b1e4b34..cea9beb17 100644 --- a/doc/unlisted/execution-thread-pool.adoc +++ b/doc/unlisted/execution-thread-pool.adoc @@ -137,8 +137,8 @@ The destructor calls `stop()` then `join()`: { thread_pool pool(4); run_async(pool.get_executor())(long_running_task()); - // Destructor: stop() -> join() -> shutdown services -> destroy services - // Queued work that hasn't started is abandoned + // Destructor: stop() -> join() -> shutdown services + // -> abandon queued work -> destroy services } ---- @@ -149,11 +149,13 @@ before the pool goes out of scope. When a pool is destroyed: -1. Workers are signaled to stop (pending work is abandoned) +1. Workers are signaled to stop 2. Worker threads are joined 3. Services are shut down (in reverse order of creation) -4. Services are destroyed -5. Remaining queued work items are destroyed +4. Remaining queued work items are abandoned (see cpp:abandon[]), + while the services still exist +5. Services are destroyed, and anything their destructors post is + abandoned in turn == Executor Operations diff --git a/example/asio/CMakeLists.txt b/example/asio/CMakeLists.txt index 003e1c5e0..646c25677 100644 --- a/example/asio/CMakeLists.txt +++ b/example/asio/CMakeLists.txt @@ -104,6 +104,7 @@ source_group(TREE ${CMAKE_CURRENT_SOURCE_DIR} PREFIX "" FILES ${PFILES}) # any_stream example (coroutine-based) add_executable(capy_example_any_stream any_stream.cpp + api/abandon_guard.hpp api/capy_streams.cpp api/capy_streams.hpp) @@ -118,6 +119,7 @@ target_link_libraries(capy_example_any_stream add_executable(capy_example_asio_callbacks asio_callbacks.cpp api/capy_streams.cpp + api/abandon_guard.hpp api/capy_streams.hpp api/uni_stream.hpp) @@ -132,6 +134,7 @@ target_link_libraries(capy_example_asio_callbacks add_executable(capy_example_use_capy use_capy_example.cpp api/capy_streams.cpp + api/abandon_guard.hpp api/capy_streams.hpp api/use_capy.hpp) diff --git a/example/asio/api/abandon_guard.hpp b/example/asio/api/abandon_guard.hpp new file mode 100644 index 000000000..4b13d5a5b --- /dev/null +++ b/example/asio/api/abandon_guard.hpp @@ -0,0 +1,86 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +#ifndef CAPY_EXAMPLE_ABANDON_GUARD_HPP +#define CAPY_EXAMPLE_ABANDON_GUARD_HPP + +#include +#include +#include + +#include + +// Rides in an Asio handler holding a parked coroutine. A handler +// destroyed without running, as when its io_context is destroyed, +// gives the coroutine up instead of leaking its chain. +class abandon_guard +{ + boost::capy::continuation* c_; + +public: + ~abandon_guard() + { + if(c_) + boost::capy::abandon(*c_); + } + + explicit abandon_guard(boost::capy::continuation& c) noexcept + : c_(&c) + { + } + + abandon_guard(abandon_guard&& other) noexcept + : c_(std::exchange(other.c_, nullptr)) + { + } + + abandon_guard(abandon_guard const&) = delete; + abandon_guard& operator=(abandon_guard const&) = delete; + abandon_guard& operator=(abandon_guard&&) = delete; + + // Return the held continuation, still given up on destruction. + boost::capy::continuation* get() const noexcept + { + return c_; + } + + // Return the continuation, which is no longer given up, or null + // if this guard holds none. + boost::capy::continuation* release() noexcept + { + return std::exchange(c_, nullptr); + } +}; + +// Resumes its coroutine when Asio runs it; gives it up when Asio +// destroys it unrun. +class resume_or_abandon +{ + abandon_guard g_; + +public: + explicit resume_or_abandon(boost::capy::continuation& c) noexcept + : g_(c) + { + } + + // Leave the coroutine to the caller, as when the post that was to + // carry it throws. Returns false if this handler was moved from. + bool release() noexcept + { + return g_.release() != nullptr; + } + + void operator()() + { + boost::capy::safe_resume(g_.release()->h); + } +}; + +#endif diff --git a/example/asio/api/capy_streams.cpp b/example/asio/api/capy_streams.cpp index 85af240c3..dd630208a 100644 --- a/example/asio/api/capy_streams.cpp +++ b/example/asio/api/capy_streams.cpp @@ -8,6 +8,7 @@ // #include "capy_streams.hpp" +#include "abandon_guard.hpp" #include #include @@ -104,21 +105,30 @@ class asio_socket capy::io_env const* env) { cont_.h = h; + cont_.owner = env->owner; cancel_ = std::make_shared(env->stop_token); - self_->socket_.async_read_some( - capy::to_asio(buffers_), - net::bind_cancellation_slot( - cancel_->signal.slot(), - [this, ex = env->executor]( - boost::system::error_code ec, - std::size_t n) mutable - { - std::get<0>(result_) = ec; - std::get<1>(result_) = n; - ex.post(cont_); - })); - + try + { + self_->socket_.async_read_some( + capy::to_asio(buffers_), + net::bind_cancellation_slot( + cancel_->signal.slot(), + [this, ex = env->executor, g = abandon_guard(cont_)]( + boost::system::error_code ec, + std::size_t n) mutable + { + std::get<0>(result_) = ec; + std::get<1>(result_) = n; + ex.post(*g.get()); + g.release(); + })); + } + catch(...) + { + // The handler was destroyed and gave the coroutine up, + // so it stays suspended for its owner to unwind. + } return std::noop_coroutine(); } @@ -168,21 +178,30 @@ class asio_socket capy::io_env const* env) { cont_.h = h; + cont_.owner = env->owner; cancel_ = std::make_shared(env->stop_token); - self_->socket_.async_write_some( - capy::to_asio(buffers_), - net::bind_cancellation_slot( - cancel_->signal.slot(), - [this, ex = env->executor]( - boost::system::error_code ec, - std::size_t n) mutable - { - std::get<0>(result_) = ec; - std::get<1>(result_) = n; - ex.post(cont_); - })); - + try + { + self_->socket_.async_write_some( + capy::to_asio(buffers_), + net::bind_cancellation_slot( + cancel_->signal.slot(), + [this, ex = env->executor, g = abandon_guard(cont_)]( + boost::system::error_code ec, + std::size_t n) mutable + { + std::get<0>(result_) = ec; + std::get<1>(result_) = n; + ex.post(*g.get()); + g.release(); + })); + } + catch(...) + { + // The handler was destroyed and gave the coroutine up, + // so it stays suspended for its owner to unwind. + } return std::noop_coroutine(); } @@ -213,21 +232,21 @@ static_assert(capy::WriteStream); class asio_executor { - net::io_context::executor_type ex_; + net::io_context* ioc_; asio_context* ctx_; public: asio_executor( - net::io_context::executor_type ex, + net::io_context* ioc, asio_context* ctx) noexcept - : ex_(ex) + : ioc_(ioc) , ctx_(ctx) { } bool operator==(asio_executor const& other) const noexcept { - return ex_ == other.ex_; + return ioc_ == other.ioc_; } asio_context& context() const noexcept @@ -240,15 +259,24 @@ class asio_executor std::coroutine_handle<> dispatch(capy::continuation& c) const { - auto h = c.h; - net::post(ex_, [h]{ h.resume(); }); + post(c); return std::noop_coroutine(); } void post(capy::continuation& c) const { - auto h = c.h; - net::post(ex_, [h]{ h.resume(); }); + resume_or_abandon r(c); + try + { + net::post(ioc_->get_executor(), std::move(r)); + } + catch(...) + { + // A handler Asio took and then destroyed has given `c` up, + // so the post must not also report it as still held. + if(r.release()) + throw; + } } }; @@ -261,12 +289,14 @@ class asio_executor struct asio_context::impl { asio_context* self; - net::io_context ioc; + // Outlives ioc: handlers it destroys give their coroutines up + // through this executor. asio_executor ex; + net::io_context ioc; explicit impl(asio_context* s) : self(s) - , ex(ioc.get_executor(), self) + , ex(&ioc, self) { } }; diff --git a/example/asio/api/uni_stream.hpp b/example/asio/api/uni_stream.hpp index 80202821c..2c11cd941 100644 --- a/example/asio/api/uni_stream.hpp +++ b/example/asio/api/uni_stream.hpp @@ -16,10 +16,12 @@ #include #include +#include "abandon_guard.hpp" #include "capy_streams.hpp" #include #include +#include #include #include @@ -57,15 +59,24 @@ class asio_executor_wrapper std::coroutine_handle<> dispatch(capy::continuation& c) const { - auto h = c.h; - net::post(ex_, [h]{ h.resume(); }); + post(c); return std::noop_coroutine(); } void post(capy::continuation& c) const { - auto h = c.h; - net::post(ex_, [h]{ h.resume(); }); + resume_or_abandon r(c); + try + { + net::post(ex_, std::move(r)); + } + catch(...) + { + // A handler Asio took and then destroyed has given `c` up, + // so the post must not also report it as still held. + if(r.release()) + throw; + } } }; diff --git a/example/asio/api/use_capy.hpp b/example/asio/api/use_capy.hpp index f0cbc2348..cda84e9ab 100644 --- a/example/asio/api/use_capy.hpp +++ b/example/asio/api/use_capy.hpp @@ -15,6 +15,8 @@ #include #include +#include "abandon_guard.hpp" + #include #include #include @@ -136,19 +138,30 @@ class capy_awaitable // post the continuation through the executor rather than resuming // inline; this restores the same-executor invariant. cont_ is a // member, giving the continuation a stable address until the - // executor dequeues and resumes it. + // executor dequeues and resumes it. The guard gives the + // coroutine up if Asio destroys the handler without running it. cont_.h = h; - auto handler = [this, ex = env->executor](Args... args) mutable + cont_.owner = env->owner; + auto handler = [this, ex = env->executor, g = abandon_guard(cont_)]( + Args... args) mutable { store_result(std::move(args)...); - ex.post(cont_); + ex.post(*g.get()); + g.release(); }; - std::move(op_)( - net::bind_cancellation_slot( - cancel_->signal.slot(), - std::move(handler))); - + try + { + std::move(op_)( + net::bind_cancellation_slot( + cancel_->signal.slot(), + std::move(handler))); + } + catch(...) + { + // The handler was destroyed and gave the coroutine up, so it + // stays suspended for its owner to unwind. + } return std::noop_coroutine(); } // end::await_suspend[] diff --git a/example/awaitable-sender/awaitable_sender_detail.hpp b/example/awaitable-sender/awaitable_sender_detail.hpp index 2f6c21a39..e4a816210 100644 --- a/example/awaitable-sender/awaitable_sender_detail.hpp +++ b/example/awaitable-sender/awaitable_sender_detail.hpp @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -134,8 +135,8 @@ constexpr bool is_compound_ec_result_v = // // The first two members match the coroutine frame layout // used by MSVC, GCC, and Clang. from_address produces a -// coroutine_handle whose .resume() calls our function -// pointer and whose .destroy() is a no-op. +// coroutine_handle whose .resume() and .destroy() call our +// function pointers. // ------------------------------------------------------- struct frame_cb @@ -243,11 +244,10 @@ class scheduler_executor using receiver_concept = ex::receiver_t; holder* self_; - std::coroutine_handle<> h_; void set_value() && noexcept { - auto h = h_; + auto h = self_->c_.h; delete self_; h.resume(); } @@ -258,12 +258,19 @@ class scheduler_executor template void set_error(E&&) && noexcept { - delete self_; + refuse(); } void set_stopped() && noexcept { + refuse(); + } + + void refuse() noexcept + { + auto c = self_->c_; delete self_; + capy::abandon(c); } }; @@ -275,12 +282,15 @@ class scheduler_executor ex::schedule(std::declval()), std::declval())); + // A copy keeps the owner, which abandon needs. + continuation c_; op_t op_; - holder(Sched s, std::coroutine_handle<> h) - : op_(ex::connect( + holder(Sched s, continuation const& c) + : c_(c) + , op_(ex::connect( ex::schedule(std::move(s)), - resume_receiver{this, h})) + resume_receiver{this})) { } }; @@ -311,7 +321,7 @@ class scheduler_executor void post(continuation& c) const { // the holder frees itself from the receiver on completion - auto* p = new holder(sched_, c.h); + auto* p = new holder(sched_, c); ex::start(p->op_); } @@ -388,10 +398,13 @@ struct awaitable_op_state return std::type_identity{}; } - Aw aw_; + // Reset to destroy the segment when it is abandoned. + std::optional aw_; Receiver rcvr_; io_env env_; frame_cb cb_; + // The segment's owner, with cb_ as its coroutine. + chain_owner owner_; // stop_cb_ declared after stop_src_ so it is destroyed first: // on_stop holds a reference to stop_src_. std::stop_source stop_src_; @@ -415,7 +428,7 @@ struct awaitable_op_state } awaitable_op_state(Aw aw, Receiver rcvr) - : aw_(std::move(aw)) + : aw_(std::in_place, std::move(aw)) , rcvr_(std::move(rcvr)) , cb_{} , stop_src_(make_stop_source()) @@ -430,11 +443,26 @@ struct awaitable_op_state static void on_resume(frame_cb* p) noexcept { auto* self = static_cast(p->data); - self->complete(); + if(self->owner_.abandoned()) + self->unwind(); + else + self->complete(); + } + + // Only giving the segment up destroys it: a closed executor + // destroying the posted unwind, or a holder with no owner to post + // to destroying it in place. + static void on_destroy(frame_cb* p) noexcept + { + static_cast(p->data)->unwind(); } - static void on_destroy(frame_cb*) noexcept + // The segment was given up: destroy its frames, and complete + // stopped, since it has no result to deliver. + void unwind() noexcept { + aw_.reset(); + ex::set_stopped(std::move(rcvr_)); } static constexpr bool nothrow_resume = @@ -471,7 +499,7 @@ struct awaitable_op_state // void and plain-value results carry no error_code, so // there is no in-band disposition; token state is the // only cancellation signal available for these rows. - aw_.await_resume(); + aw_->await_resume(); if(env_.stop_token.stop_requested()) ex::set_stopped( std::move(rcvr_)); @@ -485,7 +513,7 @@ struct awaitable_op_state // completion surfaces as stopped even if the env token // never fired, and a successful result survives a stop // request that merely landed by completion time. - auto result = aw_.await_resume(); + auto result = aw_->await_resume(); std::error_code ec; if constexpr (std::is_same_v< result_type, std::error_code>) @@ -507,7 +535,7 @@ struct awaitable_op_state } else { - auto result = aw_.await_resume(); + auto result = aw_->await_resume(); if(env_.stop_token.stop_requested()) ex::set_stopped( std::move(rcvr_)); @@ -567,8 +595,9 @@ struct awaitable_op_state } env_ = io_env{io_ex, st, fa}; + env_.owner = &owner_; - if(aw_.await_ready()) + if(aw_->await_ready()) { complete(); return; @@ -580,13 +609,14 @@ struct awaitable_op_state auto h = std::coroutine_handle<>::from_address( static_cast(&cb_)); + owner_.bind(h, io_ex); // Not a real coroutine caller, so symmetric transfer // must be driven by hand: any non-noop handle (our own // frame on immediate completion, or a wrapped task's // handle that still needs to run) has to be resumed // explicitly or nothing ever completes. - auto resumed = detail::call_await_suspend(&aw_, h, &env_); + auto resumed = detail::call_await_suspend(&*aw_, h, &env_); if(resumed != std::noop_coroutine()) resumed.resume(); } diff --git a/example/awaitable-sender/read_op.hpp b/example/awaitable-sender/read_op.hpp index 741488d39..cc04e4370 100644 --- a/example/awaitable-sender/read_op.hpp +++ b/example/awaitable-sender/read_op.hpp @@ -101,7 +101,7 @@ class read_op : public awaitable_sender_base if(stop_task_) return stop_task_->await_suspend(h, env); env_ = env; - cont_ = continuation{h}; + cont_ = continuation{h, nullptr, env->owner}; env_->executor.post(cont_); return std::noop_coroutine(); } @@ -163,7 +163,7 @@ class counted_read_op io_env const* env) { env_ = env; - cont_ = continuation{h}; + cont_ = continuation{h, nullptr, env->owner}; env_->executor.post(cont_); return std::noop_coroutine(); } diff --git a/example/awaitable-sender/tests.cpp b/example/awaitable-sender/tests.cpp index a5182f4e4..a7ddf5cb6 100644 --- a/example/awaitable-sender/tests.cpp +++ b/example/awaitable-sender/tests.cpp @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -848,6 +849,95 @@ void test_ensure_sender_lifts() CHECK(out.ch == channel::value); } +// Holds the continuation of what awaits it, as an I/O object does, +// until the test gives it up. +struct held_op +{ + capy::continuation* slot_; + std::latch* parked_; + + bool await_ready() const noexcept { return false; } + + std::coroutine_handle<> await_suspend( + std::coroutine_handle<> h, + capy::io_env const* env) noexcept + { + slot_->h = h; + slot_->owner = env->owner; + parked_->count_down(); + return std::noop_coroutine(); + } + + void await_resume() const noexcept {} +}; + +capy::task await_held(capy::continuation* slot, std::latch* parked) +{ + co_await held_op{slot, parked}; +} + +// An abandoned task delivers no result: the sender completes stopped. +void test_abandoned_task_completes_stopped() +{ + capy::thread_pool pool(1); + auto pool_ex = pool.get_executor(); + capy::continuation held; + std::latch parked(1); + std::latch done(1); + test_outcome out; + auto op = ex::connect( + capy::as_sender(await_held(&held, &parked)), + test_receiver{ + {pool_ex, {}}, &out, &done}); + ex::start(op); + WAIT_OR_DIE(parked); + capy::abandon(held); + WAIT_OR_DIE(done); + CHECK(out.ch == channel::stopped); +} + +// Parks its caller without taking the chain's owner, so giving it up +// destroys the caller in place. +struct unowned_held_op +{ + capy::continuation* slot_; + std::latch* parked_; + + bool await_ready() const noexcept { return false; } + + std::coroutine_handle<> await_suspend( + std::coroutine_handle<> h, + capy::io_env const*) noexcept + { + slot_->h = h; + parked_->count_down(); + return std::noop_coroutine(); + } + + void await_resume() const noexcept {} +}; + +// An awaitable given up in place, with no owner to unwind it, still +// completes the sender stopped. +void test_unowned_abandon_completes_stopped() +{ + capy::thread_pool pool(1); + auto pool_ex = pool.get_executor(); + capy::continuation held; + std::latch parked(1); + std::latch done(1); + test_outcome out; + auto op = ex::connect( + capy::as_sender(unowned_held_op{&held, &parked}), + test_receiver{ + {pool_ex, {}}, &out, &done}); + ex::start(op); + WAIT_OR_DIE(parked); + capy::abandon(held); + WAIT_OR_DIE(done); + CHECK(out.ch == channel::stopped); +} + // stdexec dispatches the C++26 static-template signature query; // pin read_op's deduced signatures so drift in make_sigs is // caught at compile time. @@ -869,6 +959,47 @@ static_assert(std::is_same_v< // (void)capy::as_sender(compound_op{}); // } +// Forwards to a pool, but gives up everything posted to it, as a +// context being torn down does. +struct abandoning_executor +{ + capy::thread_pool::executor_type inner_; + + bool operator==(abandoning_executor const&) const noexcept = default; + capy::execution_context& context() const noexcept + { + return inner_.context(); + } + void on_work_started() const noexcept { inner_.on_work_started(); } + void on_work_finished() const noexcept { inner_.on_work_finished(); } + std::coroutine_handle<> dispatch(capy::continuation& c) const + { + return c.h; + } + void post(capy::continuation& c) const + { + capy::abandon(c); + } +}; + +// An op given up after it posted its caller unwinds the caller's chain +// from its owner, so the launch that started it is torn down too. +template +void test_posted_op_abandon_unwinds_launch(Op op) +{ + capy::thread_pool pool(1); + auto alive = std::make_shared(); + std::weak_ptr seen = alive; + capy::run_async( + abandoning_executor{pool.get_executor()}, + [alive = std::move(alive)](auto&&...) {})( + [](Op op) -> capy::task<> + { + std::ignore = co_await std::move(op); + }(std::move(op))); + CHECK(seen.expired()); +} + int main() { test_empty_io_result_success(); @@ -905,6 +1036,12 @@ int main() test_ensure_sender_lifts(); test_read_op_cancel_std_token(); test_adaptor_immediate(); + test_abandoned_task_completes_stopped(); + test_unowned_abandon_completes_stopped(); + test_posted_op_abandon_unwinds_launch(capy::read_op::result({})); + std::size_t n = 0; + test_posted_op_abandon_unwinds_launch( + capy::counted_read_op::result({}, 0, &n)); if(failures == 0) std::cout << "all tests passed\n"; diff --git a/example/cuda/datamovement/cuda_datamovement.hpp b/example/cuda/datamovement/cuda_datamovement.hpp index d14305593..308754b6b 100644 --- a/example/cuda/datamovement/cuda_datamovement.hpp +++ b/example/cuda/datamovement/cuda_datamovement.hpp @@ -153,6 +153,7 @@ class cuda_stream return h; } self->cont_.h = h; + self->cont_.owner = env->owner; self->ctx_ = resume_ctx{env->executor, &self->cont_}; err = cudaLaunchHostFunc( self->stream_, &on_complete, &self->ctx_); @@ -187,6 +188,7 @@ class cuda_stream await_suspend(std::coroutine_handle<> h, io_env const* env) { self->cont_.h = h; + self->cont_.owner = env->owner; self->ctx_ = resume_ctx{env->executor, &self->cont_}; auto err = cudaLaunchHostFunc( self->stream_, &on_complete, &self->ctx_); @@ -345,6 +347,7 @@ class cuda_device_stream total += b.size(); } self->cont_.h = h; + self->cont_.owner = env->owner; self->ctx_ = resume_ctx{env->executor, &self->cont_}; auto err = cudaLaunchHostFunc( self->stream_, &on_complete, &self->ctx_); diff --git a/example/cuda/notification-strategies/notification_strategies.hpp b/example/cuda/notification-strategies/notification_strategies.hpp index e154b4892..81962a5e3 100644 --- a/example/cuda/notification-strategies/notification_strategies.hpp +++ b/example/cuda/notification-strategies/notification_strategies.hpp @@ -82,6 +82,7 @@ class poll_service { std::mutex mtx_; std::vector entries_; + std::vector ready_; std::jthread thread_; void run(std::stop_token st) @@ -102,11 +103,15 @@ class poll_service *e.ec = (s == cudaSuccess) ? std::error_code{} : make_cuda_error(s); - e.ex.post(*e.cont); + ready_.push_back(e); entries_[i] = entries_.back(); entries_.pop_back(); } } + // Outside the lock: a post may unwind the waiter's chain. + for(auto& e : ready_) + e.ex.post(*e.cont); + ready_.clear(); std::this_thread::yield(); } } @@ -338,6 +343,7 @@ struct callback_awaitable await_suspend(std::coroutine_handle<> h, io_env const* env) { cont_.h = h; + cont_.owner = env->owner; ctx_ = resume_ctx{env->executor, &cont_, &ec_}; auto err = cudaLaunchHostFunc(stream, &on_complete, &ctx_); if(err != cudaSuccess) @@ -399,6 +405,7 @@ class poll_awaitable await_suspend(std::coroutine_handle<> h, io_env const* env) { cont_.h = h; + cont_.owner = env->owner; svc_.register_wait(poll_entry{event_, env->executor, &cont_, &ec_}); return std::noop_coroutine(); } @@ -442,6 +449,7 @@ class deferred_sync_awaitable auto ex = env->executor; auto stream = stream_; cont_.h = h; + cont_.owner = env->owner; svc_.post([ex, stream, ec = &ec_, cont = &cont_]() mutable { auto err = cudaStreamSynchronize(stream); diff --git a/example/custom-executor/custom_executor.cpp b/example/custom-executor/custom_executor.cpp index a785cebb9..2783b39d3 100644 --- a/example/custom-executor/custom_executor.cpp +++ b/example/custom-executor/custom_executor.cpp @@ -31,7 +31,7 @@ namespace capy = boost::capy; class run_loop : public capy::execution_context { // end::inherit[] - std::queue> queue_; + std::queue queue_; std::thread::id owner_; public: @@ -46,8 +46,31 @@ class run_loop : public capy::execution_context ~run_loop() { + // Once torn down, the loop runs nothing, not even inline. + owner_ = {}; + // Nothing will run these now: give them up, so their chains + // are unwound. An unwind posted here is given up in turn. + auto drain = [this] + { + bool any = false; + while (!queue_.empty()) + { + auto c = queue_.front(); + queue_.pop(); + any = true; + capy::abandon(c); + } + return any; + }; shutdown(); - destroy(); + drain(); + // A service's destructor may post too: repeat until nothing + // is left of either. + do + { + destroy(); + } + while (drain()); } run_loop(run_loop const&) = delete; @@ -59,15 +82,16 @@ class run_loop : public capy::execution_context owner_ = std::this_thread::get_id(); while (!queue_.empty()) { - auto h = queue_.front(); + auto c = queue_.front(); queue_.pop(); - capy::safe_resume(h); + capy::safe_resume(c.h); } } - void enqueue(std::coroutine_handle<> h) + // A copy keeps the owner, which abandon needs. + void enqueue(capy::continuation const& c) { - queue_.push(h); + queue_.push(c); } bool is_running_on_this_thread() const noexcept @@ -107,14 +131,14 @@ class run_loop::executor_type { if (loop_->is_running_on_this_thread()) return c.h; // resume inline - loop_->enqueue(c.h); + loop_->enqueue(c); return std::noop_coroutine(); // defer } // end::dispatch[] void post(capy::continuation& c) const { - loop_->enqueue(c.h); + loop_->enqueue(c); } bool operator==(executor_type const& other) const noexcept diff --git a/example/gui-integration/gui_integration.cpp b/example/gui-integration/gui_integration.cpp index e33c2d328..6f2921605 100644 --- a/example/gui-integration/gui_integration.cpp +++ b/example/gui-integration/gui_integration.cpp @@ -69,8 +69,9 @@ class gui_app : public capy::execution_context std::thread::id const gui_thread_ = std::this_thread::get_id(); std::mutex m_; std::condition_variable cv_; - std::deque> queue_; + std::deque queue_; bool quit_ = false; + bool closed_ = false; public: class executor_type; @@ -82,8 +83,36 @@ class gui_app : public capy::execution_context ~gui_app() { + // Once torn down, the app runs nothing, not even inline. + closed_ = true; + // Nothing will run these now: give them up, so their chains + // are unwound. An unwind posted here is given up in turn. + auto drain = [this] + { + bool any = false; + for(;;) + { + capy::continuation c; + { + std::lock_guard lock(m_); + if(queue_.empty()) + return any; + c = queue_.front(); + queue_.pop_front(); + } + any = true; + capy::abandon(c); + } + }; shutdown(); - destroy(); + drain(); + // A service's destructor may post too: repeat until nothing + // is left of either. + do + { + destroy(); + } + while(drain()); } gui_app(gui_app const&) = delete; @@ -91,11 +120,12 @@ class gui_app : public capy::execution_context // Run a coroutine on the GUI thread. Callable from any thread and // never blocks the caller. - void post_to_gui_thread(std::coroutine_handle<> h) + // A copy keeps the owner, which abandon needs. + void post_to_gui_thread(capy::continuation const& c) { { std::lock_guard lock(m_); - queue_.push_back(h); + queue_.push_back(c); } cv_.notify_one(); } @@ -118,17 +148,17 @@ class gui_app : public capy::execution_context check(on_gui_thread(), "run() called off the GUI thread"); for(;;) { - std::coroutine_handle<> h; + capy::continuation c; { std::unique_lock lock(m_); cv_.wait(lock, [this]{ return !queue_.empty() || quit_; }); if(queue_.empty()) return; // quit requested, nothing left to run - h = queue_.front(); + c = queue_.front(); queue_.pop_front(); } - capy::safe_resume(h); + capy::safe_resume(c.h); } } // end::run[] @@ -138,6 +168,12 @@ class gui_app : public capy::execution_context return std::this_thread::get_id() == gui_thread_; } + // Whether a coroutine may be resumed inline here. + bool runs_inline() const noexcept + { + return on_gui_thread() && !closed_; + } + executor_type get_executor() noexcept; }; // end::gui_app[] @@ -168,15 +204,15 @@ class gui_app::executor_type std::coroutine_handle<> dispatch(capy::continuation& c) const { - if(app_->on_gui_thread()) + if(app_->runs_inline()) return c.h; // resume inline by symmetric transfer - app_->post_to_gui_thread(c.h); + app_->post_to_gui_thread(c); return std::noop_coroutine(); } void post(capy::continuation& c) const { - app_->post_to_gui_thread(c.h); + app_->post_to_gui_thread(c); } bool operator==(executor_type const& other) const noexcept @@ -311,6 +347,7 @@ struct show_dialog { env_ = env; cont_.h = h; + cont_.owner = env->owner; dialog_.show([this](std::string answer) { answer_ = std::move(answer); diff --git a/example/sender-bridge/sender_awaitable.hpp b/example/sender-bridge/sender_awaitable.hpp index 34d32853f..b6706f821 100644 --- a/example/sender-bridge/sender_awaitable.hpp +++ b/example/sender-bridge/sender_awaitable.hpp @@ -262,7 +262,7 @@ struct [[nodiscard]] sender_awaitable beman::execution::connect( std::move(sndr_), receiver_type{ - &result_, {h}, env})); + &result_, {h, nullptr, env->owner}, env})); op_constructed_ = true; beman::execution::start( *std::launder( diff --git a/include/boost/capy.hpp b/include/boost/capy.hpp index a385b4f48..f37fdf991 100644 --- a/include/boost/capy.hpp +++ b/include/boost/capy.hpp @@ -56,6 +56,7 @@ #include // Execution +#include #include #include #include diff --git a/include/boost/capy/concept/execution_context.hpp b/include/boost/capy/concept/execution_context.hpp index c075e86ec..73c8f8a10 100644 --- a/include/boost/capy/concept/execution_context.hpp +++ b/include/boost/capy/concept/execution_context.hpp @@ -43,8 +43,9 @@ namespace capy { resources owned by the context. @li The context remains valid while any executor referencing it exists and may be used. - @li Destroying the context abandons work submitted via associated - executors that has not started running. + @li Destroying the context discards work submitted via associated + executors that has not started running, destroying the + coroutines it would have resumed. @par Conforming Signatures diff --git a/include/boost/capy/concept/executor.hpp b/include/boost/capy/concept/executor.hpp index adf9cd385..6239e8c03 100644 --- a/include/boost/capy/concept/executor.hpp +++ b/include/boost/capy/concept/executor.hpp @@ -123,11 +123,36 @@ class execution_context; @par Executor Validity - An executor becomes invalid when the first call to - `ctx.shutdown()` returns. Calling `dispatch`, `post`, - `on_work_started`, or `on_work_finished` on an invalid executor - is undefined behavior. Copy, comparison, and `context()` remain - valid until the context is destroyed. + Once `ctx.shutdown()` has begun, the context runs no more work. + `dispatch` and `post` remain valid until the context is destroyed, + but never run the coroutine they are given: the executor passes + the continuation to @ref abandon, either at once or no later than + the context's destruction, as a context does with the coroutines + it holds. An executor takes a continuation off its queue before it + resumes or abandons it. A queued copy keeps the continuation's + `owner`, which @ref abandon needs. An abandon made inside `post` may + unwind the chain before `post` returns, so it is the executor's last + touch of the continuation, and of the executor object itself, which + may live in a frame the unwind destroys. A `post` that throws has + neither queued nor abandoned the continuation; the caller still + holds it. Copy, comparison, `context()`, + `on_work_started`, and `on_work_finished` also remain valid until + the context is destroyed. Calling any member after the context is + destroyed is undefined behavior. + + @par Throwing `post` or `dispatch` + + An exception from `post` or `dispatch` reaches the caller only + where nothing has started yet: @ref run_async rethrows it after + destroying the unstarted task. Everywhere else the library calls + them where an exception cannot propagate, and `std::terminate` is + called: while a @ref task is suspending, which includes + @ref when_all and @ref when_any starting their children and + @ref run moving to its executor; when @ref abandon posts an + unwind; when a join or a @ref run boundary resumes the + coroutine awaiting it; and when @ref async_mutex, @ref async_event + or @ref async_waker resumes a waiter, whether from an unlock, a + set, a wake, a stop request or its own destruction. @par Thread Safety Distinct objects: Safe. diff --git a/include/boost/capy/concept/io_awaitable.hpp b/include/boost/capy/concept/io_awaitable.hpp index 4c2d5a067..1c32c0c68 100644 --- a/include/boost/capy/concept/io_awaitable.hpp +++ b/include/boost/capy/concept/io_awaitable.hpp @@ -107,6 +107,13 @@ concept AwaitSuspendResult = @li Returning @c std::noop_coroutine() is equivalent to using the @c void return type, returning @c true in the @c bool return type signature. + _Teardown_: An awaitable that holds the coroutine for later, rather than + resuming it before returning, keeps it as a @ref continuation whose `owner` + is `env->owner`. If the awaitable or whatever holds the coroutine for it is + torn down first, it passes that continuation to @ref abandon, which unwinds + the chain from its owner. A continuation without an owner is destroyed + alone, leaving the frames above it holding a destroyed frame. + _Lifetime_: The object of type @ref io_env pointed to by `env` remains valid for the duration of the async operation represented by `A`. This is guarantee is maintained by @ref run, @ref run_async and the other functions that "start a task". diff --git a/include/boost/capy/concept/io_runnable.hpp b/include/boost/capy/concept/io_runnable.hpp index acc9a47bc..abd086137 100644 --- a/include/boost/capy/concept/io_runnable.hpp +++ b/include/boost/capy/concept/io_runnable.hpp @@ -79,6 +79,19 @@ namespace capy { and allocator through the coroutine chain. @li The pointed-to `io_env` must outlive the coroutine. + @par Requirements + + A type awaited through @ref run or the combinators must also take + part in the ownership protocol: + + @li The object owns the frame, and destroying the object destroys + it; a destroyed frame never destroys its continuation. A chain + given up with @ref abandon is unwound from its owner downward, + each object destroying the frame it owns. + @li Destroying the object, or assigning over it, while the frame is + lent to an awaiting chain calls `std::terminate`, unless the + chain's owner (`environment()->owner`) has been abandoned. + @par Conforming Signatures @par !example example diff --git a/include/boost/capy/continuation.hpp b/include/boost/capy/continuation.hpp index 859fe53fd..7ce7e0859 100644 --- a/include/boost/capy/continuation.hpp +++ b/include/boost/capy/continuation.hpp @@ -17,6 +17,8 @@ namespace boost { namespace capy { +class chain_owner; + /** Pairs a coroutine handle with a scratch slot so executors can queue it without heap allocation. Wraps a `std::coroutine_handle<>` with a single @@ -96,6 +98,16 @@ struct continuation class description for the full contract. */ void* reserved = nullptr; + + /** The owner of the chain segment the coroutine belongs to. + + Set from the coroutine's @ref io_env when it is parked or + queued. A holder that gives the coroutine up passes this + continuation to @ref abandon, which unwinds the segment from its + owner. Null for a coroutine with no owner, such as an owner's own + continuation, which @ref abandon destroys directly. + */ + chain_owner* owner = nullptr; }; } // namespace capy diff --git a/include/boost/capy/detail/abandonable.hpp b/include/boost/capy/detail/abandonable.hpp new file mode 100644 index 000000000..472c60524 --- /dev/null +++ b/include/boost/capy/detail/abandonable.hpp @@ -0,0 +1,76 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +#ifndef BOOST_CAPY_DETAIL_ABANDONABLE_HPP +#define BOOST_CAPY_DETAIL_ABANDONABLE_HPP + +#include +#include +#include + +#include +#include +#include + +namespace boost { +namespace capy { +namespace detail { + +/** Await an awaitable on behalf of a segment owner. + + An owner suspended here is resumed either when the awaitable + completes or to unwind its abandoned segment. In the second case the + awaitable never completed, so the result is empty (or `false` for an + awaitable with no result) and the owner must destroy the awaitable + without reading it. +*/ +template +struct abandonable +{ + using result_type = decltype(std::declval().await_resume()); + + A& a_; + chain_owner* owner_ = nullptr; + + bool await_ready() + { + return a_.await_ready(); + } + + template + auto await_suspend(H h, io_env const* env) + { + owner_ = env->owner; + return a_.await_suspend(h, env); + } + + auto await_resume() + { + bool const unwinding = owner_ && owner_->abandoned(); + if constexpr(std::is_void_v) + { + if(unwinding) + return false; + a_.await_resume(); + return true; + } + else + { + if(unwinding) + return std::optional(); + return std::optional(a_.await_resume()); + } + } +}; + +} // namespace detail +} // namespace capy +} // namespace boost + +#endif diff --git a/include/boost/capy/detail/config.hpp b/include/boost/capy/detail/config.hpp index 6ea93575c..791a239d0 100644 --- a/include/boost/capy/detail/config.hpp +++ b/include/boost/capy/detail/config.hpp @@ -139,11 +139,15 @@ #define BOOST_CAPY_CORO_AWAIT_ELIDABLE #endif -// Simpler destroy codegen for coroutines that always run to completion -#if __has_cpp_attribute(clang::coro_only_destroy_when_complete) -#define BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE [[clang::coro_only_destroy_when_complete]] +#if defined(_MSC_VER) && !defined(__clang__) +# define BOOST_CAPY_NOINLINE __declspec(noinline) +# define BOOST_CAPY_COLD +#elif defined(__GNUC__) || defined(__clang__) +# define BOOST_CAPY_NOINLINE __attribute__((noinline)) +# define BOOST_CAPY_COLD __attribute__((cold)) #else -#define BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE +# define BOOST_CAPY_NOINLINE +# define BOOST_CAPY_COLD #endif namespace boost::capy::detail { diff --git a/include/boost/capy/detail/intrusive.hpp b/include/boost/capy/detail/intrusive.hpp index b04c5fdf8..f0cb2a9a4 100644 --- a/include/boost/capy/detail/intrusive.hpp +++ b/include/boost/capy/detail/intrusive.hpp @@ -121,7 +121,7 @@ class intrusive_list remove(T* w) noexcept { if(!w->linked_) - return; + return; // LCOV_EXCL_LINE callers remove only linked nodes if(w->prev_) w->prev_->next_ = w->next_; else diff --git a/include/boost/capy/detail/owned_range.hpp b/include/boost/capy/detail/owned_range.hpp new file mode 100644 index 000000000..2c888ee46 --- /dev/null +++ b/include/boost/capy/detail/owned_range.hpp @@ -0,0 +1,59 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +#ifndef BOOST_CAPY_DETAIL_OWNED_RANGE_HPP +#define BOOST_CAPY_DETAIL_OWNED_RANGE_HPP + +#include + +#include +#include +#include +#include + +namespace boost { +namespace capy { +namespace detail { + +// A join's runners refer to their awaitables in place until they start, +// so the range it owns must yield lvalues that outlive the iteration. +// Only a forward range guarantees that: an input iterator may hold the +// element it yields and replace it on the next step. Any other range is +// moved into a vector instead. +template +using owned_range_t = std::conditional_t< + std::ranges::forward_range&> && + std::is_lvalue_reference_v< + std::ranges::range_reference_t&>>, + std::remove_cvref_t, + std::vector>>; + +template +owned_range_t +own_range(R&& r) +{ + if constexpr(std::is_same_v, std::remove_cvref_t>) + { + return std::forward(r); + } + else + { + owned_range_t v; + v.reserve(std::ranges::size(r)); + for(auto&& a : r) + v.push_back(std::move(a)); + return v; + } +} + +} // namespace detail +} // namespace capy +} // namespace boost + +#endif diff --git a/include/boost/capy/ex/abandon.hpp b/include/boost/capy/ex/abandon.hpp new file mode 100644 index 000000000..062cc63fe --- /dev/null +++ b/include/boost/capy/ex/abandon.hpp @@ -0,0 +1,136 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +#ifndef BOOST_CAPY_EX_ABANDON_HPP +#define BOOST_CAPY_EX_ABANDON_HPP + +#include +#include +#include + +#include +#include + +namespace boost { +namespace capy { + +/** The owner of one segment of a coroutine chain. + + A segment is a run of coroutines started together: the task + `run_async` launches and everything it awaits, a task `run` starts + on another executor, or one branch of `when_all` or `when_any`. Its + owner is the coroutine that started it, suspended above the + segment's top frame until that frame completes. Every coroutine in + the segment reaches the owner through its @ref io_env. + + When the holder of a suspended coroutine in the segment gives it up + with @ref abandon, the owner posts its own continuation to its + executor. There it unwinds the segment from the top: it destroys + the frame it awaits, which destroys the frames below it in turn, and + finishes without delivering a result. An executor that destroys the + posted continuation instead, as a closed one does, unwinds the + segment the same way. + + @par Thread Safety + @ref abandon may be called from any thread. Only one holder can + abandon a segment, since it is suspended at one point; a repeated + call before the unwind runs does nothing. +*/ +class chain_owner +{ + std::atomic abandoned_{false}; + continuation self_; + executor_ref ex_; + +public: + /** Bind the owner to its coroutine and executor. + + Called by the owner before it starts the segment. + + @param self The owner's coroutine, resumed or destroyed to + unwind the segment. + + @param ex The executor the segment runs on. + */ + void bind(std::coroutine_handle<> self, executor_ref ex) noexcept + { + self_.h = self; + ex_ = ex; + } + + /// Return true once the segment has been abandoned. + bool abandoned() const noexcept + { + return abandoned_.load(std::memory_order_acquire); + } + + /** Unwind the segment on the owner's executor. + + Does nothing if the segment was already abandoned. Otherwise + posts the owner's continuation; nothing of the owner is touched + afterward, since the post may unwind and free it at once. + + @note A post that throws calls `std::terminate`. + */ + void abandon() noexcept + { + if(abandoned_.exchange(true, std::memory_order_acq_rel)) + return; + ex_.post(self_); + } +}; + +/** Give up a suspended coroutine for good. + + Called by the holder of a suspended coroutine that will never resume + it, such as an execution context, strand or primitive being torn + down. The coroutine's segment is unwound from its owner: on the + owner's executor, from the top frame down to this coroutine, without + any frame delivering a result. Ordinary cancellation does not use + this; it resumes the coroutine with an error instead. + + A continuation with no owner, such as an owner's own continuation, + is destroyed directly. + + @par Preconditions + `c` refers to a suspended coroutine that the caller holds and that + nothing else will resume or abandon. If `c` has no owner, nothing + awaits the coroutine: destroying its frame alone would leave any + awaiting frame holding a destroyed one. + + @par Postconditions + The caller must not touch the coroutine, or anything in its frame, + again. The frames may already be destroyed when this returns, if the + owner's executor destroys posted work at once. + + @note The unwind runs on the owner's executor, which may belong to + another execution context than the caller's, and may run after the + caller's context is destroyed. A chain that holds objects of the + caller's context must therefore have finished before that context + is torn down. + + @note Each frame is destroyed from the destructor of the frame + awaiting it, so the unwind uses stack in proportion to how deeply + the chain's coroutines are nested. + + @param c The continuation of the coroutine given up. +*/ +inline void +abandon(continuation& c) noexcept +{ + if(c.owner) + c.owner->abandon(); + else if(c.h) + c.h.destroy(); +} + +} // namespace capy +} // namespace boost + +#endif diff --git a/include/boost/capy/ex/async_event.hpp b/include/boost/capy/ex/async_event.hpp index d0fb7a2bd..d57fa84b7 100644 --- a/include/boost/capy/ex/async_event.hpp +++ b/include/boost/capy/ex/async_event.hpp @@ -38,10 +38,12 @@ claims. Waiters already claimed by a stop callback are skipped. Because set() pops all waiters, a canceled waiter may have been - removed from the list by set() before its await_resume runs. - This requires a separate in_list_ flag (unlike async_mutex where - active_ served double duty). await_resume only calls remove() - when in_list_ is true. + removed from the list by set() before its await_resume runs. Each + waiter therefore records the list it is linked into, if any: the + event's, or set()'s own while set() posts the waiters it claimed. + A waiter destroyed or canceled while linked unlinks itself from + that list, so a post that tears down one claimed waiter, and with + it another, never leaves set() holding a freed node. */ namespace boost { @@ -93,6 +95,9 @@ class async_event bool set_ = false; detail::intrusive_list waiters_; + // Unlink every waiter and move the ones this call claims into into. + void claim_all(detail::intrusive_list& into) noexcept; + public: /** Suspends the caller until `set()` runs, or resumes it with `error::canceled` on a stop request. */ @@ -111,7 +116,7 @@ class async_event std::atomic claimed_{false}; bool canceled_ = false; bool active_ = false; - bool in_list_ = false; + detail::intrusive_list* list_ = nullptr; struct cancel_fn { @@ -159,8 +164,8 @@ class async_event { if(active_) stop_cb_().~stop_cb_t(); - if(in_list_) - e_->waiters_.remove(this); + if(list_) + list_->remove(this); } /** Construct an awaiter for the given event. @@ -188,7 +193,7 @@ class async_event std::memory_order_relaxed)) , canceled_(o.canceled_) , active_(std::exchange(o.active_, false)) - , in_list_(std::exchange(o.in_list_, false)) + , list_(std::exchange(o.list_, nullptr)) { } @@ -266,9 +271,10 @@ class async_event return h; } cont_.h = h; + cont_.owner = env->owner; ex_ = env->executor; e_->waiters_.push_back(this); - in_list_ = true; + list_ = &e_->waiters_; ::new(stop_cb_buf_) stop_cb_t( env->stop_token, cancel_fn{this}); active_ = true; @@ -294,10 +300,10 @@ class async_event } if(canceled_) { - if(in_list_) + if(list_) { - e_->waiters_.remove(this); - in_list_ = false; + list_->remove(this); + list_ = nullptr; } return {make_error_code( error::canceled)}; @@ -341,6 +347,27 @@ class async_event */ async_event& operator=(async_event&& other) = delete; + /** Destroy the event, canceling every coroutine still waiting. + + Each waiter is posted to its executor to resume with + `error::canceled`, as if its stop token had been triggered. An + executor that can no longer run it abandons it instead, possibly + before this destructor returns. Either way the waiter no longer + refers to the event. + */ + ~async_event() + { + while(auto* w = waiters_.pop_front()) + { + w->list_ = nullptr; + if(!w->claimed_.exchange(true, std::memory_order_acq_rel)) + { + w->canceled_ = true; + w->ex_.post(w->cont_); + } + } + } + /** Returns an awaiter that waits until the event is set. If the event is already set, completes immediately. @@ -357,21 +384,21 @@ class async_event All waiting coroutines are resumed. Canceled waiters are skipped. Subsequent calls to wait() complete immediately until clear() is called. + + @note If posting a waiter's resumption throws, `std::terminate` + is called. */ - void set() + void set() noexcept { set_ = true; - for(;;) + // A post can destroy a waiter on the spot, and with it whatever + // owns this event; nothing below touches *this. + detail::intrusive_list claimed; + claim_all(claimed); + while(auto* w = claimed.pop_front()) { - auto* w = waiters_.pop_front(); - if(!w) - break; - w->in_list_ = false; - if(!w->claimed_.exchange( - true, std::memory_order_acq_rel)) - { - w->ex_.post(w->cont_); - } + w->list_ = nullptr; + w->ex_.post(w->cont_); } } @@ -395,6 +422,21 @@ class async_event } }; +inline void +async_event::claim_all( + detail::intrusive_list& into) noexcept +{ + while(auto* w = waiters_.pop_front()) + { + w->list_ = nullptr; + if(!w->claimed_.exchange(true, std::memory_order_acq_rel)) + { + into.push_back(w); + w->list_ = &into; + } + } +} + } // namespace capy } // namespace boost diff --git a/include/boost/capy/ex/async_mutex.hpp b/include/boost/capy/ex/async_mutex.hpp index b73d5f861..43973d89e 100644 --- a/include/boost/capy/ex/async_mutex.hpp +++ b/include/boost/capy/ex/async_mutex.hpp @@ -138,6 +138,17 @@ class async_mutex private: bool locked_ = false; detail::intrusive_list waiters_; + // Handed the lock but not yet resumed; the destructor must reach it. + lock_awaiter* granted_to_ = nullptr; + + // State of the unlock() in progress, on its own stack: a post can + // destroy the granted waiter, re-entering unlock(), or this mutex. + struct unlock_scope + { + bool again = false; + bool gone = false; + }; + unlock_scope* unlocking_ = nullptr; public: /** Suspends the caller until the mutex is free, or resumes it with `error::canceled` on a stop request. @@ -157,6 +168,13 @@ class async_mutex bool canceled_ = false; bool active_ = false; + // Set by unlock() when it hands the lock to this waiter. + bool granted_ = false; + + // Cleared by whoever pops the node, so a waiter outliving + // its mutex never reaches through m_. + bool in_list_ = false; + struct cancel_fn { lock_awaiter* self_; @@ -198,6 +216,10 @@ class async_mutex the stop callback can then reach a destroyed awaiter when the coroutine frame is torn down while suspended. + If the lock was handed to this awaiter and the coroutine is + destroyed before it resumes, the lock is released to the next + waiter. + @par Preconditions Called on the executor thread. The stop callback may fire from any thread, so destroying a still-suspended awaiter from @@ -208,7 +230,14 @@ class async_mutex if(active_) { stop_cb_().~stop_cb_t(); + } + if(in_list_) m_->waiters_.remove(this); + // Handed the lock but destroyed before resuming: pass it on. + if(granted_) + { + m_->granted_to_ = nullptr; + m_->unlock(); } } @@ -237,6 +266,8 @@ class async_mutex std::memory_order_relaxed)) , canceled_(o.canceled_) , active_(std::exchange(o.active_, false)) + , granted_(std::exchange(o.granted_, false)) + , in_list_(std::exchange(o.in_list_, false)) { } @@ -324,8 +355,10 @@ class async_mutex return h; } cont_.h = h; + cont_.owner = env->owner; ex_ = env->executor; m_->waiters_.push_back(this); + in_list_ = true; ::new(stop_cb_buf_) stop_cb_t( env->stop_token, cancel_fn{this}); active_ = true; @@ -339,17 +372,23 @@ class async_mutex @return An empty `io_result<>` if the mutex is now held by the awaiting coroutine. Otherwise one holding `error::canceled`, - which means the stop token won the race and the mutex is not - held. + which means the stop token won the race or the mutex was + destroyed, and the mutex is not held. */ [[nodiscard]] io_result<> await_resume() noexcept { + if(std::exchange(granted_, false)) + m_->granted_to_ = nullptr; if(active_) { stop_cb_().~stop_cb_t(); if(canceled_) { - m_->waiters_.remove(this); + if(in_list_) + { + m_->waiters_.remove(this); + in_list_ = false; + } active_ = false; return {make_error_code( error::canceled)}; @@ -522,6 +561,40 @@ class async_mutex /// Construct an unlocked mutex. async_mutex() = default; + /** Destroy the mutex, canceling every coroutine still waiting. + + Each waiter is posted to its executor to resume with + `error::canceled`, as if its stop token had been triggered. An + executor that can no longer run it abandons it instead, possibly + before this destructor returns. Either way the waiter no longer + refers to the mutex. + + A waiter handed the lock but not yet resumed resumes with + `error::canceled` instead, holding nothing. + + @pre No coroutine holds the lock. + */ + ~async_mutex() + { + if(unlocking_) + unlocking_->gone = true; + // Already posted, so it is only told the lock no longer exists. + if(granted_to_) + { + granted_to_->granted_ = false; + granted_to_->canceled_ = true; + } + while(auto* w = waiters_.pop_front()) + { + w->in_list_ = false; + if(!w->claimed_.exchange(true, std::memory_order_acq_rel)) + { + w->canceled_ = true; + w->ex_.post(w->cont_); + } + } + } + /** Copy construction is disabled; suspended waiters point into the mutex's wait queue. @@ -581,23 +654,49 @@ class async_mutex */ void unlock() noexcept { - for(;;) + if(unlocking_) { - auto* waiter = waiters_.pop_front(); + // Reached from a granted waiter destroyed by the post below; + // that call hands the lock on once the post returns. + unlocking_->again = true; + return; + } + unlock_scope scope; + unlocking_ = &scope; + do + { + scope.again = false; + auto* waiter = grant_next(); if(!waiter) - { - locked_ = false; + break; + waiter->ex_.post(waiter->cont_); + if(scope.gone) return; - } + } + while(scope.again); + unlocking_ = nullptr; + } + +private: + // Hand the lock to the next unclaimed waiter, or release it. + lock_awaiter* grant_next() noexcept + { + while(auto* waiter = waiters_.pop_front()) + { + waiter->in_list_ = false; if(!waiter->claimed_.exchange( true, std::memory_order_acq_rel)) { - waiter->ex_.post(waiter->cont_); - return; + waiter->granted_ = true; + granted_to_ = waiter; + return waiter; } } + locked_ = false; + return nullptr; } +public: /** Returns true if the mutex is currently locked. @return `true` if the mutex is held; otherwise `false`. diff --git a/include/boost/capy/ex/async_waker.hpp b/include/boost/capy/ex/async_waker.hpp index 61531a056..47e25fcbd 100644 --- a/include/boost/capy/ex/async_waker.hpp +++ b/include/boost/capy/ex/async_waker.hpp @@ -22,6 +22,7 @@ #include #include #include +#include #include /* async_waker implementation notes @@ -33,9 +34,10 @@ waker has to dereference the waiter, and nothing would pin the waiter's frame between reading the pointer and claiming it. - So the three-state st_ atomic is the single arbiter: + So the st_ atomic is the single arbiter: - empty --arm--> armed --wake/cancel CAS--> empty + empty --arm--> arming --publish--> armed --wake/cancel CAS--> empty + arming --stop already requested--> empty empty --wake--> token --wait consumes--> empty Whoever wins the armed->empty CAS owns the resume and may @@ -45,17 +47,18 @@ callback wins, a concurrent wake retries, finds empty, and latches a token -- a racing wakeup is deferred, never lost. - Serialized resumption is required: await_suspend keeps - writing after the publishing armed-CAS (the stop_cb - placement-new and active_ = true), so a wake/cancel winner - can post the continuation while that tail is still running. - The posted resume must be ordered after await_suspend's - return, which holds on a single-threaded executor (the one - thread is still inside await_suspend) and on a strand (the - resume is a later turn, synchronized with the current one). - A raw multi-threaded executor lets another worker run - await_resume against those in-flight writes. async_event and - async_mutex make the same assumption; it is stated explicitly + The stop callback is registered before the waiter enters the + arming state, and a wake() winner destroys it before posting. + While arming, nothing can claim the waiter: wake(), the stop + callback and the destructor wait the few instructions it takes. + So the waiter settles a stop requested during registration + itself, its last touch of the waker is the publishing store, + and the waker can die right after the wake() that claimed it. + + Serialized resumption is still required: the executor must not + run the waiter's continuations concurrently, since a winner can + post the waiter before its await_suspend has returned. async_event + and async_mutex make the same assumption; it is stated explicitly here because wake() invites foreign threads into the picture. */ @@ -109,13 +112,34 @@ class async_waker class wait_awaiter; private: - static constexpr int state_empty = 0; // no token, no waiter - static constexpr int state_token = 1; // latched wakeup - static constexpr int state_armed = 2; // waiter suspended + static constexpr int state_empty = 0; // no token, no waiter + static constexpr int state_token = 1; // latched wakeup + static constexpr int state_armed = 2; // waiter suspended + static constexpr int state_arming = 3; // waiter publishing itself std::atomic st_{state_empty}; wait_awaiter* waiter_ = nullptr; + // Take the armed waiter, waiting out one still publishing itself. + bool claim_armed() noexcept + { + int expected = state_armed; + for(;;) + { + if(st_.compare_exchange_weak( + expected, state_empty, + std::memory_order_acq_rel, + std::memory_order_acquire)) + return true; + if(expected == state_armed) + continue; // LCOV_EXCL_LINE spurious weak CAS failure + if(expected != state_arming) + return false; + std::this_thread::yield(); // LCOV_EXCL_LINE race with a waiter still publishing itself + expected = state_armed; // LCOV_EXCL_LINE race with a waiter still publishing itself + } + } + public: /** Suspends the caller until `wake()` runs, or resumes it with `error::canceled` on a stop request. */ @@ -134,19 +158,23 @@ class async_waker bool active_ = false; bool published_ = false; + // Set by whoever wins the armed->empty transition, before it + // posts. A claimed waiter belongs to the executor's queue and + // may be discarded after the waker is gone, so its destructor + // must not touch the waker. + std::atomic claimed_{false}; + struct cancel_fn { wait_awaiter* self_; void operator()() const noexcept { - int expected = state_armed; - if(self_->waker_->st_.compare_exchange_strong( - expected, state_empty, - std::memory_order_acq_rel, - std::memory_order_acquire)) + if(self_->waker_->claim_armed()) { self_->canceled_ = true; + self_->claimed_.store( + true, std::memory_order_release); self_->ex_.post(self_->cont_); } } @@ -182,7 +210,8 @@ class async_waker { if(active_) stop_cb_().~stop_cb_t(); - if(published_) + if(published_ && + !claimed_.load(std::memory_order_acquire)) { // Destroyed while still armed (frame torn down // without resuming): deregister so a later @@ -219,6 +248,7 @@ class async_waker , canceled_(o.canceled_) , active_(std::exchange(o.active_, false)) , published_(std::exchange(o.published_, false)) + , claimed_(o.claimed_.load(std::memory_order_relaxed)) { } @@ -282,12 +312,15 @@ class async_waker `await_ready` looked, so the token is consumed here instead and the wait succeeds. - @li Otherwise the slot moves to the armed state, publishing this - awaiter to the waker, and a stop callback is registered on - `env->stop_token`. Whichever of `wake()` and that callback - wins the armed-to-empty transition posts `h` through - `env->executor`. The loser does nothing, and a losing - `wake()` re-latches its token for the next `wait()`. + @li Otherwise a stop callback is registered on + `env->stop_token` and the slot moves to the armed state, + publishing this awaiter to the waker. A stop requested + while the callback registered completes the wait canceled + instead. Whichever of `wake()` and that callback wins the + armed-to-empty transition posts `h` through + `env->executor`; a winning `wake()` first deregisters the + callback. The loser does nothing, and a losing `wake()` + re-latches its token for the next `wait()`. @param h The awaiting coroutine, resumed when the waker fires or the wait is canceled. @@ -312,12 +345,20 @@ class async_waker return h; } cont_.h = h; + cont_.owner = env->owner; ex_ = env->executor; waker_->waiter_ = this; + // Registered before publishing, so the wake() that claims + // this waiter can deregister it before posting. + ::new(stop_cb_buf_) stop_cb_t( + env->stop_token, cancel_fn{this}); + active_ = true; + published_ = true; + int expected = state_empty; if(!waker_->st_.compare_exchange_strong( - expected, state_armed, + expected, state_arming, std::memory_order_acq_rel, std::memory_order_acquire)) { @@ -327,15 +368,26 @@ class async_waker // A wake latched between await_ready and here; // consume it and resume inline. + published_ = false; + stop_cb_().~stop_cb_t(); + active_ = false; waker_->st_.store( state_empty, std::memory_order_release); return h; } - published_ = true; - ::new(stop_cb_buf_) stop_cb_t( - env->stop_token, cancel_fn{this}); - active_ = true; + // While arming nothing else can claim this waiter, so a stop + // requested as the callback registered is settled here, and + // the waker is not touched again once the waiter is armed. + if(env->stop_token.stop_requested()) + { + published_ = false; + waker_->st_.store( + state_empty, std::memory_order_release); + canceled_ = true; + return h; + } + waker_->st_.store(state_armed, std::memory_order_release); return std::noop_coroutine(); } @@ -347,7 +399,8 @@ class async_waker @return An empty `io_result<>` if the wait was woken, whether by `wake()` or by a token consumed inline. Otherwise one holding - `error::canceled`, which means the stop token won the race. + `error::canceled`, which means the stop token won the race or + the waker was destroyed. */ [[nodiscard]] io_result<> await_resume() noexcept { @@ -366,6 +419,35 @@ class async_waker /// Construct with no token latched. async_waker() = default; + /** Destroy the waker, canceling a coroutine still waiting. + + The waiter is posted to its executor to resume with + `error::canceled`, as if its stop token had been triggered. An + executor that can no longer run it abandons it instead, possibly + before this destructor returns. Either way the waiter no longer + refers to the waker. + + @pre No `wake()` call is in progress or made afterward. + */ + ~async_waker() + { + if(claim_armed()) + { + auto* w = waiter_; + // The stop callback reaches into the waker; deregistering + // it waits out one already running. + if(w->active_) + { + using cb_t = wait_awaiter::stop_cb_t; + w->stop_cb_().~cb_t(); + w->active_ = false; + } + w->canceled_ = true; + w->claimed_.store(true, std::memory_order_release); + w->ex_.post(w->cont_); + } + } + /** Copy construction is disabled; an armed waiter points into the waker. @@ -403,7 +485,8 @@ class async_waker fires. @par Preconditions - No other coroutine is currently waiting on this object. + No other coroutine is currently waiting on this object; a + coroutine woken or canceled but not yet resumed still is. @return An awaitable that await-returns `io_result<>`; empty on wakeup, `error::canceled` if the stop @@ -419,7 +502,9 @@ class async_waker Callable from any thread. The waiter's resumption is posted through its executor; this call never resumes a coroutine inline. Multiple calls without an intervening - `wait()` collapse into a single token. + `wait()` collapse into a single token. It may wait briefly + for a waiter that is still suspending, or for a stop callback + of the waiter that is running on another thread. */ void wake() noexcept { @@ -437,6 +522,13 @@ class async_waker return; continue; } + // The waiter is still publishing itself; it settles in a + // few instructions. + if(s == state_arming) + { + std::this_thread::yield(); + continue; + } // armed: winning this CAS claims the waiter, whose // frame is pinned until we post its resumption. if(st_.compare_exchange_weak( @@ -445,6 +537,13 @@ class async_waker std::memory_order_acquire)) { auto* w = waiter_; + // Nothing may reach this waker once the waiter is + // posted; this waits out a stop callback already + // running, which then finds the slot empty. + using cb_t = wait_awaiter::stop_cb_t; + w->stop_cb_().~cb_t(); + w->active_ = false; + w->claimed_.store(true, std::memory_order_release); w->ex_.post(w->cont_); return; } diff --git a/include/boost/capy/ex/execution_context.hpp b/include/boost/capy/ex/execution_context.hpp index 8cc7e15ee..b36dc78c0 100644 --- a/include/boost/capy/ex/execution_context.hpp +++ b/include/boost/capy/ex/execution_context.hpp @@ -37,7 +37,11 @@ namespace capy { Derived classes such as `io_context` extend this to provide execution facilities like event loops and thread pools. Derived class destructors must call `shutdown()` and `destroy()` to ensure - proper service cleanup before member destruction. + proper service cleanup before member destruction. Between the two, + a context that queues work abandons what is still queued. Since a + service's destructor can post and an unwound chain can create a + service, it then repeats `destroy()` and that drain until the + drain finds nothing. @par Service Lifecycle Services are created on first use via `use_service()` or explicitly @@ -449,7 +453,9 @@ class BOOST_CAPY_DECL subsequent calls have no effect. @par Effects - Each service's `shutdown()` member function is invoked once. + Each service's `shutdown()` member function is invoked once. A + service created after this call has its `shutdown()` invoked as + soon as it is created, so it too starts in a stopped state. @par Postconditions @li All services are in a stopped state. @@ -471,9 +477,15 @@ class BOOST_CAPY_DECL @par Preconditions @li `shutdown()` was called. + @li No service's destructor creates a service of a type + already destroyed here; two such destructors that re-create + each other never let this return. @par Effects - All services are deleted and removed from the container. + All services are deleted and removed from the container. Each + is removed just before it is deleted, so a destructor can still + find the services not yet destroyed. A service a destructor + creates is deleted in turn. @par Postconditions @li The service container is empty. diff --git a/include/boost/capy/ex/frame_alloc_mixin.hpp b/include/boost/capy/ex/frame_alloc_mixin.hpp index e4b0d4127..c4f1bea94 100644 --- a/include/boost/capy/ex/frame_alloc_mixin.hpp +++ b/include/boost/capy/ex/frame_alloc_mixin.hpp @@ -116,6 +116,30 @@ struct frame_alloc_mixin } }; +namespace detail { + +// For capy's own frames whose teardown can release the chain that owns +// the frame allocator: they are never allocated from it, so freeing them +// afterwards never reaches a resource that died with that chain. +struct process_frame_mixin +{ + static void* operator new(std::size_t size) + { + return static_cast( + get_recycling_memory_resource()) + ->allocate_fast(size, alignof(std::max_align_t)); + } + + static void operator delete(void* ptr, std::size_t size) noexcept + { + static_cast( + get_recycling_memory_resource()) + ->deallocate_fast(ptr, size, alignof(std::max_align_t)); + } +}; + +} // namespace detail + } // namespace capy } // namespace boost diff --git a/include/boost/capy/ex/frame_allocator.hpp b/include/boost/capy/ex/frame_allocator.hpp index 2d66c6a26..e82e08063 100644 --- a/include/boost/capy/ex/frame_allocator.hpp +++ b/include/boost/capy/ex/frame_allocator.hpp @@ -37,6 +37,31 @@ namespace capy { namespace detail { +#if (defined(BOOST_CAPY_DYN_LINK) || defined(BOOST_ALL_DYN_LINK)) && \ + !defined(BOOST_CAPY_STATIC_LINK) + +/** Return this thread's frame allocator slot. + + A shared library build defines it in the library: an inline + definition gives each module its own slot, and a coroutine created + in one would not see the allocator a launcher in another installed. + A static build keeps it inline, since resuming a coroutine writes + it. + + @return A reference to the calling thread's slot. +*/ +BOOST_CAPY_DECL +std::pmr::memory_resource*& +current_frame_allocator_ref() noexcept; + +#else + +// MSVC may keep an inlined thread_local's address in a coroutine +// frame across a suspension, so a resumption on another thread +// would write the first thread's slot. +#if BOOST_CAPY_WORKAROUND(_MSC_VER, >= 1) && !defined(__clang__) +BOOST_CAPY_NOINLINE +#endif inline std::pmr::memory_resource*& current_frame_allocator_ref() noexcept { @@ -44,6 +69,8 @@ current_frame_allocator_ref() noexcept return mr; } +#endif + } // namespace detail /** Return the current frame allocator for this thread. diff --git a/include/boost/capy/ex/io_awaitable_promise_base.hpp b/include/boost/capy/ex/io_awaitable_promise_base.hpp index 216d3f30b..ffcc7df68 100644 --- a/include/boost/capy/ex/io_awaitable_promise_base.hpp +++ b/include/boost/capy/ex/io_awaitable_promise_base.hpp @@ -12,6 +12,7 @@ #define BOOST_CAPY_EX_IO_AWAITABLE_PROMISE_BASE_HPP #include +#include #include #include #include @@ -72,11 +73,29 @@ namespace capy { @par !example example_3 + @par Ownership + A coroutine's frame is owned by the object that awaits it, such as + the parent's `task`, so a chain is destroyed from its top down, each + frame freeing the one it awaits. The suspended coroutine at the + bottom is held by whoever will resume it: an operation, queue or + primitive. A holder that will never resume it calls @ref abandon, + which unwinds the chain's segment from its owner (see + @ref chain_owner) rather than destroying the bottom frame. + Destroying a frame that is still awaited, other than in that + unwind, calls `std::terminate`. + @par Thread Safety The environment is stored during `await_suspend` and read during `co_await this_coro::environment`. These occur on the same logical thread of execution, so no synchronization is required. + @note A chain waiting on a primitive stored in its own frames owns + itself through that primitive; it is freed only by a stop request or + by the primitive being signalled. A mutex must outlive every + coroutine holding or handed its lock. A cross-thread `wake()` racing + the teardown of an armed waker's frame on its executor thread is a + precondition violation. + @see this_coro::environment, this_coro::executor, this_coro::stop_token, this_coro::frame_allocator @see io_env @@ -90,32 +109,6 @@ class io_awaitable_promise_base mutable std::coroutine_handle<> cont_{std::noop_coroutine()}; public: - /** Destroy the promise, destroying an orphaned continuation. - - A continuation is still stored only when the coroutine never - reached `final_suspend`, because @ref continuation consumes the - stored handle. Destroying it here is what keeps an abandoned - coroutine from leaking the trampoline frame that was waiting on it. - - @par Preconditions - No parent coroutine is awaiting this one. A parent's `await_suspend` - installs its own handle as the continuation, so destroying such a - coroutine directly would destroy the parent from here as well. See - @ref task::handle and @ref quitter::handle for the contract. - */ - ~io_awaitable_promise_base() - { - // Abnormal teardown: destroy an orphaned continuation, e.g. - // a run_async trampoline when the task is destroyed before - // reaching final_suspend. Callers must not destroy a task - // via handle().destroy() while it is being awaited by a - // parent coroutine: that puts cont_ under another owner - // and would produce a double-destroy from this branch. See - // task::handle() / quitter::handle() for the contract. - if(cont_ != std::noop_coroutine()) - cont_.destroy(); - } - //---------------------------------------------------------- // Continuation support //---------------------------------------------------------- diff --git a/include/boost/capy/ex/io_env.hpp b/include/boost/capy/ex/io_env.hpp index b3d9d8b00..ff7d7ec28 100644 --- a/include/boost/capy/ex/io_env.hpp +++ b/include/boost/capy/ex/io_env.hpp @@ -21,6 +21,8 @@ namespace boost { namespace capy { +class chain_owner; + /** Carries the executor, stop token, and frame allocator through a coroutine chain. This struct bundles the execution context passed through @@ -55,6 +57,23 @@ struct io_env */ std::pmr::memory_resource* frame_allocator = nullptr; + /** The owner of the chain segment running in this environment. + + Set by whatever started the segment (`run_async`, `run` on + another executor, a `when_all` or `when_any` branch). An + awaitable that parks or queues a coroutine copies it into the + @ref continuation it keeps, so a holder can @ref abandon the + segment. Null when nothing owns the segment. + + A launcher that starts a task under an environment it builds + itself, rather than through `run_async`, `run` or a join, must + either own the segment, binding a @ref chain_owner, or ensure + nothing awaits the coroutines that could be abandoned. With no + owner, @ref abandon destroys only the frame it is given, and any + frame awaiting it is left holding a destroyed frame. + */ + chain_owner* owner = nullptr; + }; } // capy diff --git a/include/boost/capy/ex/recycling_memory_resource.hpp b/include/boost/capy/ex/recycling_memory_resource.hpp index 486b792e3..a9c5c0406 100644 --- a/include/boost/capy/ex/recycling_memory_resource.hpp +++ b/include/boost/capy/ex/recycling_memory_resource.hpp @@ -112,11 +112,19 @@ class BOOST_CAPY_DECL recycling_memory_resource : public std::pmr::memory_resour // global()'s holder destructor at process exit. }; +#if (defined(BOOST_CAPY_DYN_LINK) || defined(BOOST_ALL_DYN_LINK)) && \ + !defined(BOOST_CAPY_STATIC_LINK) + // A shared build defines this thread's pool, and the flag below, in + // the library: inline, each module would cache in a pool of its own, + // which the library's thread-exit cleanup never drains. + static pool& local() noexcept; +#else static pool& local() noexcept { static thread_local pool p; return p; } +#endif static pool& global() noexcept; static std::mutex& global_mutex() noexcept; @@ -126,9 +134,23 @@ class BOOST_CAPY_DECL recycling_memory_resource : public std::pmr::memory_resour // Register a thread-exit callback that drains this thread's local // pool back to the OS. Called only off the hot path: unconditionally - // from the slow paths, and once per thread from deallocate_fast - // behind a guard-free flag. - static void arm_thread_cleanup() noexcept; + // from the slow paths, and from deallocate_fast while the flag below + // is clear. Returns false once that callback has run: nothing drains + // the local pool after it, so frees must bypass it. + static bool arm_thread_cleanup() noexcept; + + // Guard-free flag (constinit bool, trivial dtor): set once cleanup is + // armed, and cleared again when the cleanup has run. +#if (defined(BOOST_CAPY_DYN_LINK) || defined(BOOST_ALL_DYN_LINK)) && \ + !defined(BOOST_CAPY_STATIC_LINK) + static bool& thread_armed() noexcept; +#else + static bool& thread_armed() noexcept + { + static thread_local bool armed = false; + return armed; + } +#endif public: /** Destroy the resource. @@ -202,14 +224,18 @@ class BOOST_CAPY_DECL recycling_memory_resource : public std::pmr::memory_resour ::operator delete(p); return; } - // Guard-free flag (constinit bool, trivial dtor): arms thread-exit - // cleanup exactly once for any thread that caches via deallocate, - // including consumer threads that never hit a slow path. - static thread_local bool armed = false; + // Arms thread-exit cleanup for any thread that caches via + // deallocate, including consumer threads that never hit a slow + // path; after that cleanup, the block bypasses the local pool. + auto& armed = thread_armed(); if(!armed) { + if(!arm_thread_cleanup()) + { + deallocate_slow(p, idx); + return; + } armed = true; - arm_thread_cleanup(); } auto& lp = local(); if(lp.buckets[idx].push(p)) diff --git a/include/boost/capy/ex/run.hpp b/include/boost/capy/ex/run.hpp index 43bc56330..7b0d0eda1 100644 --- a/include/boost/capy/ex/run.hpp +++ b/include/boost/capy/ex/run.hpp @@ -15,12 +15,14 @@ #include #include #include +#include #include #include #include #include #include +#include #include #include #include @@ -35,27 +37,17 @@ co_await run(ex, alloc)(my_task()); - The evaluation order is: - 1. run(ex, alloc) creates a temporary wrapper - 2. my_task() allocates its coroutine frame using TLS - 3. operator() returns an awaitable - 4. Wrapper temporary is DESTROYED - 5. co_await suspends caller, resumes task - 6. Task body executes (wrapper is already dead!) - - Problem: The wrapper's frame_memory_resource dies before the task - body runs. When initial_suspend::await_resume() restores TLS from - the saved pointer, it would point to dead memory. - - Solution: Store a COPY of the allocator in the awaitable (not just - the wrapper). The co_await mechanism extends the awaitable's lifetime - until the await completes. In await_suspend, we overwrite the promise's - saved frame_allocator pointer to point to the awaitable's resource. - - This works because standard allocator copies are equivalent - memory - allocated with one copy can be deallocated with another copy. The - task's own frame uses the footer-stored pointer (safe), while nested - task creation uses TLS pointing to the awaitable's resource (also safe). + The wrapper run(ex, alloc) returns is a temporary holding the + frame_memory_resource my_task()'s frame is allocated from, so it must + live until that frame is freed: the awaitable is awaited in the + full-expression that made the wrapper, and the temporary lives until + the co_await completes. Keeping the awaitable and awaiting it later + frees the task's frame through a dead resource. + + The awaitable also holds a copy of the allocator. In await_suspend it + overwrites the promise's saved frame_allocator pointer with the + awaitable's resource, so tasks the inner task creates allocate from + it through TLS. */ namespace boost::capy::detail { @@ -65,18 +57,52 @@ namespace boost::capy::detail { Sits between the inner task and the parent when executors diverge. The inner task's `final_suspend` resumes this trampoline via symmetric transfer. The trampoline's own - `final_suspend` dispatches the parent through the caller's - executor to restore the correct execution context. - - The trampoline never touches the task's result. + `final_suspend` posts the parent through the caller's executor to + restore the correct execution context. + + It also owns the inner task's segment. Abandoning that segment + resumes the trampoline on the inner executor, or has a closed + executor destroy it: either way it destroys the inner task, which + unwinds the segment, then abandons the caller's segment, which + unwinds the parent on its own executor. The caller's execution + context must outlive that. */ -struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE boundary_trampoline +struct boundary_trampoline { struct promise_type - : frame_alloc_mixin + : detail::process_frame_mixin { executor_ref caller_ex_; continuation parent_; + chain_owner segment_; + // The awaitable's trampoline object until the inner task + // completes or is unwound; null afterward. + boundary_trampoline* owner_ = nullptr; + // Destroys the inner task the awaitable owns. + void (*destroy_inner_)(void*) noexcept = nullptr; + void* awaitable_ = nullptr; + + // Unwind the inner segment, then hand the parent to its own + // segment's owner. The last touch of this frame: abandoning the + // parent can destroy the awaitable, and this frame with it. + void unwind() noexcept + { + destroy_inner_(awaitable_); + owner_ = nullptr; + auto parent = parent_; + capy::abandon(parent); + } + + ~promise_type() + { + // Linked only when a closed executor destroyed this frame + // instead of resuming it to unwind. + if(owner_) + { + owner_->h_ = nullptr; + unwind(); + } + } boundary_trampoline get_return_object() noexcept { @@ -96,7 +122,18 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE boundary_trampoline auto await_suspend( std::coroutine_handle<>) noexcept { - p_->caller_ex_.post(p_->parent_); + // Nothing here touches p_ after the post or the + // unwind: either can destroy this frame. + if(p_->segment_.abandoned()) + { + p_->unwind(); + } + else + { + p_->owner_ = nullptr; + auto ex = p_->caller_ex_; + ex.post(p_->parent_); + } return detail::symmetric_transfer( std::noop_coroutine()); } @@ -116,7 +153,11 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE boundary_trampoline ~boundary_trampoline() { - if(h_) h_.destroy(); + if(h_) + { + h_.promise().owner_ = nullptr; + h_.destroy(); + } } boundary_trampoline(boundary_trampoline const&) = delete; @@ -129,7 +170,11 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE boundary_trampoline { if(this != &o) { - if(h_) h_.destroy(); + if(h_) + { + h_.promise().owner_ = nullptr; + h_.destroy(); + } h_ = std::exchange(o.h_, nullptr); } return *this; @@ -171,10 +216,33 @@ struct [[nodiscard]] run_awaitable_ex frame_memory_resource resource_; std::conditional_t st_; io_env env_; + executor_ref caller_ex_; boundary_trampoline tr_; continuation task_cont_; + bool holds_work_ = false; Task inner_; // Last: destroyed first, while env_ is still valid + ~run_awaitable_ex() + { + // Only a parent destroyed from above leaves the trampoline + // still linked: the inner task is still running. + if(tr_.h_ && tr_.h_.promise().owner_) + std::terminate(); // LCOV_EXCL_LINE reached only in a death test's child process + // A parent unwound instead of resumed never reaches + // await_resume. + if(holds_work_) + caller_ex_.on_work_finished(); + } + + // Called by the trampoline to unwind the inner segment. + static void destroy_inner(void* p) noexcept + { + auto& self = *static_cast(p); + auto h = self.inner_.handle(); + self.inner_.release(); + h.destroy(); + } + // void allocator, inherit stop token run_awaitable_ex(Ex ex, Task inner) requires (InheritStopToken && std::is_void_v) @@ -220,19 +288,34 @@ struct [[nodiscard]] run_awaitable_ex decltype(auto) await_resume() { + if(std::exchange(holds_work_, false)) + caller_ex_.on_work_finished(); return inner_.await_resume(); } std::coroutine_handle<> await_suspend(std::coroutine_handle<> cont, io_env const* caller_env) { tr_ = make_boundary_trampoline(); - tr_.h_.promise().caller_ex_ = caller_env->executor; - tr_.h_.promise().parent_.h = cont; + auto& tp = tr_.h_.promise(); + tp.owner_ = &tr_; + tp.caller_ex_ = caller_env->executor; + tp.parent_.h = cont; + tp.parent_.owner = caller_env->owner; + tp.destroy_inner_ = &run_awaitable_ex::destroy_inner; + tp.awaitable_ = this; + tp.segment_.bind(tr_.h_, ex_); + caller_ex_ = caller_env->executor; + // Released once the parent resumes or is unwound. Not on the + // post back: the parent's chain may own the executor object and + // can be gone by the time post returns. + caller_ex_.on_work_started(); + holds_work_ = true; auto h = inner_.handle(); auto& p = h.promise(); p.set_continuation(tr_.h_); + env_.owner = &tp.segment_; env_.executor = ex_; if constexpr (InheritStopToken) env_.stop_token = caller_env->stop_token; @@ -246,7 +329,22 @@ struct [[nodiscard]] run_awaitable_ex p.set_environment(&env_); task_cont_.h = h; - ex_.post(task_cont_); + task_cont_.owner = &tp.segment_; + try + { + ex_.post(task_cont_); + } + catch(...) + { + // The parent resumes with the exception: nothing may still + // treat the task as lent or hold work for the parent. + p.set_continuation(std::noop_coroutine()); + task_cont_.h = nullptr; + tr_ = boundary_trampoline(); + holds_work_ = false; + caller_ex_.on_work_finished(); + throw; + } return std::noop_coroutine(); } @@ -276,8 +374,21 @@ struct [[nodiscard]] run_awaitable frame_memory_resource resource_; std::conditional_t st_; io_env env_; + // Set while the parent is suspended awaiting the inner task. + bool lent_ = false; Task inner_; // Last: destroyed first, while env_ is still valid + ~run_awaitable() + { + // Only the unwind of an abandoned chain destroys an inner task + // still awaited; checked here so it terminates whatever the + // task type does when destroyed while lent. + auto h = inner_.handle(); + if(lent_ && h && !h.done() && + !(env_.owner && env_.owner->abandoned())) + std::terminate(); // LCOV_EXCL_LINE reached only in a death test's child process + } + // void allocator, inherit stop token explicit run_awaitable(Task inner) requires (InheritStopToken && std::is_void_v) @@ -319,6 +430,7 @@ struct [[nodiscard]] run_awaitable decltype(auto) await_resume() { + lent_ = false; return inner_.await_resume(); } @@ -327,7 +439,9 @@ struct [[nodiscard]] run_awaitable auto h = inner_.handle(); auto& p = h.promise(); p.set_continuation(cont); + lent_ = true; + env_.owner = caller_env->owner; env_.executor = caller_env->executor; if constexpr (InheritStopToken) env_.stop_token = caller_env->stop_token; @@ -366,6 +480,10 @@ class [[nodiscard]] run_wrapper_ex frame_memory_resource resource_; Alloc alloc_; // Copy to pass to awaitable + std::pmr::memory_resource* saved_tls_ = + get_current_frame_allocator(); + bool restored_ = false; + public: run_wrapper_ex(Ex ex, Alloc alloc) requires InheritStopToken @@ -386,6 +504,15 @@ class [[nodiscard]] run_wrapper_ex set_current_frame_allocator(&resource_); } + // Restores the caller's allocator when evaluating the task argument + // threw. Not afterwards: a wrapper destroyed with a torn-down parent + // frame must not touch the tearing thread's allocator. + ~run_wrapper_ex() + { + if(!restored_) + set_current_frame_allocator(saved_tls_); + } + // Non-copyable, non-movable (must be used immediately) run_wrapper_ex(run_wrapper_ex const&) = delete; run_wrapper_ex(run_wrapper_ex&&) = delete; @@ -395,6 +522,9 @@ class [[nodiscard]] run_wrapper_ex template [[nodiscard]] auto operator()(Task t) && { + // The task's frame is made; the caller's own frames are not. + set_current_frame_allocator(saved_tls_); + restored_ = true; if constexpr (InheritStopToken) return run_awaitable_ex{ std::move(ex_), std::move(alloc_), std::move(t)}; @@ -412,6 +542,10 @@ class [[nodiscard]] run_wrapper_ex st_; std::pmr::memory_resource* mr_; + std::pmr::memory_resource* saved_tls_ = + get_current_frame_allocator(); + bool restored_ = false; + public: run_wrapper_ex(Ex ex, std::pmr::memory_resource* mr) requires InheritStopToken @@ -430,6 +564,15 @@ class [[nodiscard]] run_wrapper_ex [[nodiscard]] auto operator()(Task t) && { + // The task's frame is made; the caller's own frames are not. + set_current_frame_allocator(saved_tls_); + restored_ = true; if constexpr (InheritStopToken) return run_awaitable_ex{ std::move(ex_), mr_, std::move(t)}; @@ -499,6 +645,10 @@ class [[nodiscard]] run_wrapper frame_memory_resource resource_; Alloc alloc_; // Copy to pass to awaitable + std::pmr::memory_resource* saved_tls_ = + get_current_frame_allocator(); + bool restored_ = false; + public: explicit run_wrapper(Alloc alloc) requires InheritStopToken @@ -517,6 +667,15 @@ class [[nodiscard]] run_wrapper set_current_frame_allocator(&resource_); } + // Restores the caller's allocator when evaluating the task argument + // threw. Not afterwards: a wrapper destroyed with a torn-down parent + // frame must not touch the tearing thread's allocator. + ~run_wrapper() + { + if(!restored_) + set_current_frame_allocator(saved_tls_); + } + // Non-copyable, non-movable (must be used immediately) run_wrapper(run_wrapper const&) = delete; run_wrapper(run_wrapper&&) = delete; @@ -526,6 +685,9 @@ class [[nodiscard]] run_wrapper template [[nodiscard]] auto operator()(Task t) && { + // The task's frame is made; the caller's own frames are not. + set_current_frame_allocator(saved_tls_); + restored_ = true; if constexpr (InheritStopToken) return run_awaitable{ std::move(alloc_), std::move(t)}; @@ -542,6 +704,10 @@ class [[nodiscard]] run_wrapper std::conditional_t st_; std::pmr::memory_resource* mr_; + std::pmr::memory_resource* saved_tls_ = + get_current_frame_allocator(); + bool restored_ = false; + public: explicit run_wrapper(std::pmr::memory_resource* mr) requires InheritStopToken @@ -558,6 +724,15 @@ class [[nodiscard]] run_wrapper set_current_frame_allocator(mr_); } + // Restores the caller's allocator when evaluating the task argument + // threw. Not afterwards: a wrapper destroyed with a torn-down parent + // frame must not touch the tearing thread's allocator. + ~run_wrapper() + { + if(!restored_) + set_current_frame_allocator(saved_tls_); + } + // Non-copyable, non-movable (must be used immediately) run_wrapper(run_wrapper const&) = delete; run_wrapper(run_wrapper&&) = delete; @@ -567,6 +742,9 @@ class [[nodiscard]] run_wrapper template [[nodiscard]] auto operator()(Task t) && { + // The task's frame is made; the caller's own frames are not. + set_current_frame_allocator(saved_tls_); + restored_ = true; if constexpr (InheritStopToken) return run_awaitable{ mr_, std::move(t)}; @@ -610,6 +788,16 @@ namespace boost::capy { Returns a wrapper that accepts a task and produces an awaitable. When co_awaited, the task runs on the specified executor. + If the inner task is abandoned before it completes, for example + because its execution context is torn down, the caller's chain is + abandoned too: it is unwound by its owner, on the owner's executor, + without the caller resuming. The awaitable holds work on the + caller's executor until the caller resumes or is unwound. + + @par Preconditions + The caller's execution context outlives the inner task and the + unwind of the caller's chain. This is not detected. + @par Example @par !example example_2 @@ -634,6 +822,8 @@ run(Ex ex) @param st The stop token for cooperative cancellation. @return A wrapper that accepts a task for execution. + + @see run(Ex), which describes teardown and its precondition. */ template [[nodiscard]] auto @@ -649,6 +839,8 @@ run(Ex ex, std::stop_token st) @param mr The memory resource for frame allocation. @return A wrapper that accepts a task for execution. + + @see run(Ex), which describes teardown and its precondition. */ template [[nodiscard]] auto @@ -660,10 +852,16 @@ run(Ex ex, std::pmr::memory_resource* mr) /** Bind a task to an executor with a standard allocator. + Await the awaitable in the expression that creates it, as in + `co_await run(ex, alloc)(task())`: the task's frame is allocated from + memory the wrapper holds until the end of that expression. + @param ex The executor on which the task should run. @param alloc The allocator for frame allocation. @return A wrapper that accepts a task for execution. + + @see run(Ex), which describes teardown and its precondition. */ template [[nodiscard]] auto @@ -680,6 +878,8 @@ run(Ex ex, Alloc alloc) @param mr The memory resource for frame allocation. @return A wrapper that accepts a task for execution. + + @see run(Ex), which describes teardown and its precondition. */ template [[nodiscard]] auto @@ -691,11 +891,17 @@ run(Ex ex, std::stop_token st, std::pmr::memory_resource* mr) /** Bind a task to an executor with stop token and standard allocator. + Await the awaitable in the expression that creates it, as in + `co_await run(ex, alloc)(task())`: the task's frame is allocated from + memory the wrapper holds until the end of that expression. + @param ex The executor on which the task should run. @param st The stop token for cooperative cancellation. @param alloc The allocator for frame allocation. @return A wrapper that accepts a task for execution. + + @see run(Ex), which describes teardown and its precondition. */ template [[nodiscard]] auto @@ -749,6 +955,10 @@ run(std::pmr::memory_resource* mr) // documentation. Keep the `@param`. /** Run a task with a custom standard allocator. + Await the awaitable in the expression that creates it, as in + `co_await run(alloc)(task())`: the task's frame is allocated from + memory the wrapper holds until the end of that expression. + The task inherits the caller's executor. The allocator is used for nested frame allocations. @@ -781,6 +991,10 @@ run(std::stop_token st, std::pmr::memory_resource* mr) /** Run a task with stop token and standard allocator. + Await the awaitable in the expression that creates it, as in + `co_await run(alloc)(task())`: the task's frame is allocated from + memory the wrapper holds until the end of that expression. + The task inherits the caller's executor. @param st The stop token for cooperative cancellation. diff --git a/include/boost/capy/ex/run_async.hpp b/include/boost/capy/ex/run_async.hpp index 137959555..2b63f8b12 100644 --- a/include/boost/capy/ex/run_async.hpp +++ b/include/boost/capy/ex/run_async.hpp @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -97,7 +98,7 @@ struct get_promise_awaiter @tparam Alloc The allocator type (value type or memory_resource*). */ template -struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE run_async_trampoline +struct run_async_trampoline { using invoke_fn = void(*)(void*, Handlers&); @@ -114,6 +115,17 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE run_async_trampoline // Both must reference the same coroutine and be kept in sync. std::coroutine_handle<> task_h_; continuation task_cont_; + // Owns the task's segment: abandoning it resumes this + // trampoline, or has it destroyed, to unwind the task. + chain_owner owner_; + + ~promise_type() + { + // Still owned only when a closed executor destroyed this + // trampoline instead of resuming it to unwind the task. + if(task_h_) + task_h_.destroy(); + } promise_type(Ex& ex, Handlers& h, Alloc& a) noexcept : wg_(std::move(ex)) @@ -211,8 +223,7 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE run_async_trampoline This avoids double indirection when the user passes a memory_resource*. */ template -struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE - run_async_trampoline +struct run_async_trampoline { using invoke_fn = void(*)(void*, Handlers&); @@ -229,6 +240,17 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE // Both must reference the same coroutine and be kept in sync. std::coroutine_handle<> task_h_; continuation task_cont_; + // Owns the task's segment: abandoning it resumes this + // trampoline, or has it destroyed, to unwind the task. + chain_owner owner_; + + ~promise_type() + { + // Still owned only when a closed executor destroyed this + // trampoline instead of resuming it to unwind the task. + if(task_h_) + task_h_.destroy(); + } promise_type( Ex& ex, Handlers& h, std::pmr::memory_resource* mr) noexcept @@ -315,10 +337,13 @@ make_trampoline(Ex, Handlers, Alloc) struct frame_guard { std::coroutine_handle<>& h; - ~frame_guard() { h.destroy(); } + ~frame_guard() { std::exchange(h, nullptr).destroy(); } } guard{p.task_h_}; - p.invoke_(p.task_promise_, p.handlers_); + // Resumed with the task unfinished: the segment was abandoned, and + // destroying the task unwinds it without calling a handler. + if(!p.owner_.abandoned()) + p.invoke_(p.task_promise_, p.handlers_); } } // namespace detail @@ -400,7 +425,8 @@ class [[nodiscard]] run_async_wrapper @note When `Alloc` is not `std::pmr::memory_resource*` it must be nothrow move constructible (enforced by a `static_assert`), which - is what allows this constructor to be `noexcept`. + is what allows this constructor to be `noexcept`. If allocating + the launch state throws, `std::terminate` is called. */ run_async_wrapper( Ex ex, @@ -427,9 +453,14 @@ class [[nodiscard]] run_async_wrapper Resets the thread-local frame allocator to the value saved at construction. A stale pointer to the trampoline's resource therefore does not outlive the execution context that owns it. + If no task was started, for example because evaluating the task + argument threw, the unused launch state is destroyed, releasing + the executor's work. */ ~run_async_wrapper() { + if(tr_.h_) + tr_.h_.destroy(); set_current_frame_allocator(saved_tls_); } @@ -474,6 +505,10 @@ class [[nodiscard]] run_async_wrapper in its chain. Awaitables may store `io_env const*` without concern for dangling references. + @throws Any exception thrown by the executor's `dispatch`. The + task is then destroyed without running, and the executor's + work is released. + @tparam Task The IoRunnable type. @param t The task to execute. Ownership is transferred to the @@ -495,14 +530,21 @@ class [[nodiscard]] run_async_wrapper // Setup task's continuation to return to run_async_trampoline task_promise.set_continuation(tr_.h_); - p.env_ = {p.wg_.executor(), st_, p.get_resource()}; + p.owner_.bind(tr_.h_, p.wg_.executor()); + p.env_ = {p.wg_.executor(), st_, p.get_resource(), &p.owner_}; task_promise.set_environment(&p.env_); // Start task through executor. // safe_resume is not needed here: TLS is already saved in the // constructor (saved_tls_) and restored in the destructor. p.task_cont_.h = task_h; - p.wg_.executor().dispatch(p.task_cont_).resume(); + p.task_cont_.owner = &p.owner_; + // If dispatch throws, the destructor destroys the trampoline + // and the unstarted task with it. + auto next = p.wg_.executor().dispatch(p.task_cont_); + // From here the trampoline destroys itself when the task ends. + tr_.h_ = nullptr; + next.resume(); } }; @@ -555,6 +597,8 @@ run_async(Ex ex) The handler `h1` is called with the task's result on success. If `h1` is also invocable with `std::exception_ptr`, it handles exceptions too. Otherwise, an unhandled exception calls `std::terminate`. + If the task is abandoned before it completes, as a context's teardown + does, no handler is called; the handlers are destroyed with it. Construct the task as the direct argument of the two-call expression `run_async(ex)(task)`. @@ -593,6 +637,8 @@ run_async(Ex ex, H1 h1) The handler `h1` is called with the task's result on success. The handler `h2` is called with the exception_ptr on failure. + If the task is abandoned before it completes, as a context's teardown + does, no handler is called; the handlers are destroyed with it. Construct the task as the direct argument of the two-call expression `run_async(ex)(task)`. diff --git a/include/boost/capy/ex/strand.hpp b/include/boost/capy/ex/strand.hpp index 646f79428..e05f4352c 100644 --- a/include/boost/capy/ex/strand.hpp +++ b/include/boost/capy/ex/strand.hpp @@ -64,6 +64,9 @@ namespace capy { itself is always safe, including after the context is destroyed. + Coroutines queued on the strand when its context shuts down are + passed to @ref abandon rather than resumed. + @par Thread Safety Distinct objects: Safe. Shared objects: Safe. diff --git a/include/boost/capy/ex/thread_pool.hpp b/include/boost/capy/ex/thread_pool.hpp index d0e20afe1..e5a905caf 100644 --- a/include/boost/capy/ex/thread_pool.hpp +++ b/include/boost/capy/ex/thread_pool.hpp @@ -132,8 +132,9 @@ class BOOST_CAPY_DECL /** Request all worker threads to stop. Signals all threads to exit after finishing their current - work item. Queued work that has not started is abandoned. - Does not wait for threads to exit. + work item. Queued work that has not started stays queued + until the pool is destroyed, which abandons it. Does not + wait for threads to exit. If @ref join is blocking on another thread, calling `stop()` causes it to stop waiting for outstanding diff --git a/include/boost/capy/io/any_read_stream.hpp b/include/boost/capy/io/any_read_stream.hpp index 229055280..cc0ee26f0 100644 --- a/include/boost/capy/io/any_read_stream.hpp +++ b/include/boost/capy/io/any_read_stream.hpp @@ -80,12 +80,22 @@ class any_read_stream void* cached_awaitable_ = nullptr; void* storage_ = nullptr; bool awaitable_active_ = false; + // The flag of the awaitable whose operation is active, cleared + // when the stream ends that operation itself. + bool* owner_ = nullptr; + + void + disown() noexcept + { + if(owner_) + *std::exchange(owner_, nullptr) = false; + } public: /** Destructor. - Destroys the owned stream (if any) and releases the cached - awaitable storage. + Ends any operation in flight, then destroys the owned stream + (if any) and releases the cached awaitable storage. */ ~any_read_stream(); @@ -130,6 +140,7 @@ class any_read_stream , storage_(std::exchange(other.storage_, nullptr)) , awaitable_active_(std::exchange(other.awaitable_active_, false)) { + other.disown(); } /** Assign by moving. @@ -220,6 +231,16 @@ class any_read_stream read_some(MB buffers); protected: + /** End the operation in flight, if any. + + Destroys the wrapped stream's active operation, which may refer + to the stream, so an owner must call this before destroying the + stream. The awaitable that started the operation no longer ends + it. + */ + void + end_operation() noexcept; + /** Rebind to a new stream after move. Updates the internal pointer to reference a new stream object. @@ -301,20 +322,28 @@ struct any_read_stream::vtable_for_impl }; }; +inline void +any_read_stream::end_operation() noexcept +{ + disown(); + if(awaitable_active_) + { + vt_->destroy_awaitable(cached_awaitable_); + awaitable_active_ = false; + } +} + inline any_read_stream::~any_read_stream() { + end_operation(); + if(cached_awaitable_) + ::operator delete(cached_awaitable_); if(storage_) { vt_->destroy(stream_); ::operator delete(storage_); } - if(cached_awaitable_) - { - if(awaitable_active_) - vt_->destroy_awaitable(cached_awaitable_); - ::operator delete(cached_awaitable_); - } } inline any_read_stream& @@ -322,17 +351,15 @@ any_read_stream::operator=(any_read_stream&& other) noexcept { if(this != &other) { + end_operation(); + other.disown(); + if(cached_awaitable_) + ::operator delete(cached_awaitable_); if(storage_) { vt_->destroy(stream_); ::operator delete(storage_); } - if(cached_awaitable_) - { - if(awaitable_active_) - vt_->destroy_awaitable(cached_awaitable_); - ::operator delete(cached_awaitable_); - } stream_ = std::exchange(other.stream_, nullptr); vt_ = std::exchange(other.vt_, nullptr); cached_awaitable_ = std::exchange(other.cached_awaitable_, nullptr); @@ -391,6 +418,38 @@ any_read_stream::read_some(MB buffers) { any_read_stream* self_; detail::mutable_buffer_array ba_; + // Set while this awaitable, not another on the same stream, + // holds the stream's inner awaitable. + bool owns_ = false; + + // A chain unwound while suspended here never reaches + // await_resume, so the inner awaitable is destroyed here. + ~awaitable() + { + if(owns_) + { + self_->vt_->destroy_awaitable(self_->cached_awaitable_); + self_->awaitable_active_ = false; + self_->owner_ = nullptr; + } + } + + awaitable( + any_read_stream* self, + detail::mutable_buffer_array ba) noexcept + : self_(self) + , ba_(ba) + { + } + + awaitable(awaitable&& other) noexcept + : self_(std::exchange(other.self_, nullptr)) + , ba_(other.ba_) + , owns_(std::exchange(other.owns_, false)) + { + if(owns_) + self_->owner_ = &owns_; + } bool await_ready() @@ -400,6 +459,8 @@ any_read_stream::read_some(MB buffers) self_->cached_awaitable_, ba_.to_span()); self_->awaitable_active_ = true; + owns_ = true; + self_->owner_ = &owns_; return self_->vt_->await_ready( self_->cached_awaitable_); @@ -416,12 +477,15 @@ any_read_stream::read_some(MB buffers) await_resume() { struct guard { - any_read_stream* self; + awaitable* a; ~guard() { - self->vt_->destroy_awaitable(self->cached_awaitable_); - self->awaitable_active_ = false; + a->self_->vt_->destroy_awaitable( + a->self_->cached_awaitable_); + a->self_->awaitable_active_ = false; + a->self_->owner_ = nullptr; + a->owns_ = false; } - } g{self_}; + } g{this}; return self_->vt_->await_resume( self_->cached_awaitable_); } diff --git a/include/boost/capy/io/any_stream.hpp b/include/boost/capy/io/any_stream.hpp index 33fb3fcb4..b00639a8e 100644 --- a/include/boost/capy/io/any_stream.hpp +++ b/include/boost/capy/io/any_stream.hpp @@ -68,11 +68,14 @@ class any_stream public: /** Destructor. - Destroys the owned stream (if any). Base class destructors - handle their cached awaitable storage. + Ends any operation in flight, then destroys the owned stream + (if any). Base class destructors release their cached awaitable + storage. */ ~any_stream() { + any_read_stream::end_operation(); + any_write_stream::end_operation(); if(storage_) { destroy_(stream_ptr_); @@ -134,6 +137,8 @@ class any_stream { if(this != &other) { + any_read_stream::end_operation(); + any_write_stream::end_operation(); if(storage_) { destroy_(stream_ptr_); diff --git a/include/boost/capy/io/any_write_stream.hpp b/include/boost/capy/io/any_write_stream.hpp index efe681d23..170e0d8be 100644 --- a/include/boost/capy/io/any_write_stream.hpp +++ b/include/boost/capy/io/any_write_stream.hpp @@ -81,12 +81,22 @@ class any_write_stream void* cached_awaitable_ = nullptr; void* storage_ = nullptr; bool awaitable_active_ = false; + // The flag of the awaitable whose operation is active, cleared + // when the stream ends that operation itself. + bool* owner_ = nullptr; + + void + disown() noexcept + { + if(owner_) + *std::exchange(owner_, nullptr) = false; + } public: /** Destructor. - Destroys the owned stream (if any) and releases the cached - awaitable storage. + Ends any operation in flight, then destroys the owned stream + (if any) and releases the cached awaitable storage. */ ~any_write_stream(); @@ -131,6 +141,7 @@ class any_write_stream , storage_(std::exchange(other.storage_, nullptr)) , awaitable_active_(std::exchange(other.awaitable_active_, false)) { + other.disown(); } /** Assign by moving. @@ -223,6 +234,16 @@ class any_write_stream write_some(CB buffers); protected: + /** End the operation in flight, if any. + + Destroys the wrapped stream's active operation, which may refer + to the stream, so an owner must call this before destroying the + stream. The awaitable that started the operation no longer ends + it. + */ + void + end_operation() noexcept; + /** Rebind to a new stream after move. Updates the internal pointer to reference a new stream object. @@ -304,20 +325,28 @@ struct any_write_stream::vtable_for_impl }; }; +inline void +any_write_stream::end_operation() noexcept +{ + disown(); + if(awaitable_active_) + { + vt_->destroy_awaitable(cached_awaitable_); + awaitable_active_ = false; + } +} + inline any_write_stream::~any_write_stream() { + end_operation(); + if(cached_awaitable_) + ::operator delete(cached_awaitable_); if(storage_) { vt_->destroy(stream_); ::operator delete(storage_); } - if(cached_awaitable_) - { - if(awaitable_active_) - vt_->destroy_awaitable(cached_awaitable_); - ::operator delete(cached_awaitable_); - } } inline any_write_stream& @@ -325,17 +354,15 @@ any_write_stream::operator=(any_write_stream&& other) noexcept { if(this != &other) { + end_operation(); + other.disown(); + if(cached_awaitable_) + ::operator delete(cached_awaitable_); if(storage_) { vt_->destroy(stream_); ::operator delete(storage_); } - if(cached_awaitable_) - { - if(awaitable_active_) - vt_->destroy_awaitable(cached_awaitable_); - ::operator delete(cached_awaitable_); - } stream_ = std::exchange(other.stream_, nullptr); vt_ = std::exchange(other.vt_, nullptr); cached_awaitable_ = std::exchange(other.cached_awaitable_, nullptr); @@ -390,6 +417,21 @@ any_write_stream::write_some(CB buffers) { any_write_stream* self_; detail::const_buffer_array ba_; + // Set while this awaitable, not another on the same stream, + // holds the stream's inner awaitable. + bool owns_ = false; + + // A chain unwound while suspended here never reaches + // await_resume, so the inner awaitable is destroyed here. + ~awaitable() + { + if(owns_) + { + self_->vt_->destroy_awaitable(self_->cached_awaitable_); + self_->awaitable_active_ = false; + self_->owner_ = nullptr; + } + } awaitable( any_write_stream* self, @@ -399,6 +441,15 @@ any_write_stream::write_some(CB buffers) { } + awaitable(awaitable&& other) noexcept + : self_(std::exchange(other.self_, nullptr)) + , ba_(other.ba_) + , owns_(std::exchange(other.owns_, false)) + { + if(owns_) + self_->owner_ = &owns_; + } + bool await_ready() { @@ -411,6 +462,8 @@ any_write_stream::write_some(CB buffers) self_->cached_awaitable_, ba_.to_span()); self_->awaitable_active_ = true; + owns_ = true; + self_->owner_ = &owns_; return self_->vt_->await_ready( self_->cached_awaitable_); @@ -426,15 +479,18 @@ any_write_stream::write_some(CB buffers) [[nodiscard]] io_result await_resume() { - if(!self_->awaitable_active_) + if(!owns_) return {std::error_code(), 0}; struct guard { - any_write_stream* self; + awaitable* a; ~guard() { - self->vt_->destroy_awaitable(self->cached_awaitable_); - self->awaitable_active_ = false; + a->self_->vt_->destroy_awaitable( + a->self_->cached_awaitable_); + a->self_->awaitable_active_ = false; + a->self_->owner_ = nullptr; + a->owns_ = false; } - } g{self_}; + } g{this}; return self_->vt_->await_resume( self_->cached_awaitable_); } diff --git a/include/boost/capy/io/write_now.hpp b/include/boost/capy/io/write_now.hpp index d11cfc945..2372ea2c8 100644 --- a/include/boost/capy/io/write_now.hpp +++ b/include/boost/capy/io/write_now.hpp @@ -18,6 +18,8 @@ #include #include #include +#include +#include #include #include @@ -61,6 +63,10 @@ namespace capy { `operator()` must not be made until the previous operation has completed (i.e., the returned awaitable is fully consumed). + Destroying the returned operation object while it is being awaited + calls `std::terminate`, except while its chain is unwound after an + @ref abandon. + @par Example @code @@ -98,6 +104,9 @@ class write_now io_env const* env_ = nullptr; bool done_ = false; + // The awaiting op_type while a parent awaits this frame. + op_type* owner_ = nullptr; + op_type get_return_object() { return op_type{ @@ -170,15 +179,23 @@ class write_now return inner_.await_ready(); } - std::coroutine_handle<> await_suspend(std::coroutine_handle<> h) + // The inner operation may resume or destroy this + // frame on another thread before control returns + // here, so the handle must not round-trip through + // the frame. + auto await_suspend(std::coroutine_handle<> h) { - return detail::call_await_suspend( - &inner_, h, - p_->env_); + return detail::symmetric_transfer( + detail::call_await_suspend( + &inner_, h, + p_->env_)); } decltype(auto) await_resume() { + // The resumption may arrive on another thread. + set_current_frame_allocator( + p_->env_->frame_allocator); return inner_.await_resume(); } }; @@ -219,7 +236,15 @@ class write_now ~op_type() { if(h_) + { + // Only the unwind of an abandoned chain destroys a frame + // that is still awaited. + auto& p = h_.promise(); + if(!p.done_ && p.owner_ && + !(p.env_->owner && p.env_->owner->abandoned())) + std::terminate(); // LCOV_EXCL_LINE reached only in a death test's child process h_.destroy(); + } } op_type(op_type const&) = delete; @@ -244,12 +269,14 @@ class write_now auto& p = h_.promise(); p.cont_ = cont; p.env_ = env; + p.owner_ = this; return h_; } [[nodiscard]] io_result await_resume() { auto& p = h_.promise(); + p.owner_ = nullptr; if(p.ep_) std::rethrow_exception(p.ep_); return p.result_; diff --git a/include/boost/capy/quitter.hpp b/include/boost/capy/quitter.hpp index ecbd46f73..f89f6f1d1 100644 --- a/include/boost/capy/quitter.hpp +++ b/include/boost/capy/quitter.hpp @@ -465,11 +465,18 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE */ std::coroutine_handle h_; - /// Destroy the quitter and its coroutine frame if owned. + /** Destroy the quitter and its coroutine frame if owned. + + @par Preconditions + The frame is not lent: no parent is suspended awaiting it, + unless its chain is being unwound after an @ref abandon. + Destroying a lent frame otherwise calls `std::terminate`; end + a running chain by requesting stop, or by destroying its + execution context. + */ ~quitter() { - if(h_) - h_.destroy(); + destroy_frame(); } /** Return false; quitters are never immediately ready. @@ -500,6 +507,7 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE */ auto await_resume() { + lent_ = false; if(h_.promise().stopped()) throw detail::stop_requested_exception{}; if(h_.promise().state_ == promise_type::completion::exception) @@ -531,17 +539,26 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE { h_.promise().set_continuation(cont); h_.promise().set_environment(env); + lent_ = true; return h_; } /** Return the coroutine handle. - @note Do not call `destroy()` on the returned handle while - the quitter is being awaited. The quitter's lifetime is - normally managed by `run_async`, `run`, or the awaiting - parent. Manually destroying a suspended quitter that another - coroutine is awaiting produces undefined behavior. For - cooperative cancellation, use `std::stop_token`. + @note Do not call `destroy()` on the returned handle while the + quitter is being awaited: the awaiting frame still holds it and + destroys it again, which has undefined behavior. A holder that + gives the coroutine up passes its continuation to @ref abandon + instead. (Destroying or + move-assigning over the quitter object itself while its frame is + lent is a different operation, and calls `std::terminate`.) + The quitter's lifetime is normally managed by `run_async`, `run`, + or the awaiting parent. For cooperative cancellation, use + `std::stop_token`. + + @note The operation, queue or primitive holding a suspended + coroutine gives it up by passing the continuation it keeps to + @ref abandon. `handle()` belongs to the owning quitter object. @return The coroutine handle. */ @@ -554,8 +571,8 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE @note The caller may call `destroy()` on the released handle only when the quitter has not started or has fully completed. - Destroying a suspended quitter that is being awaited produces - undefined behavior. + Destroying the released handle of a quitter that is being + awaited is not supported and has undefined behavior. @par Postconditions `handle()` returns a null handle. Callers needing the @@ -598,6 +615,10 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE If this quitter already owns a coroutine frame, that frame is destroyed first. Self-assignment is a no-op. + @par Preconditions + The frame being replaced is not lent: no parent is suspended + awaiting it. Replacing a lent frame calls `std::terminate`. + @par Postconditions `other` is empty and must not be awaited. @@ -609,18 +630,36 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE { if(this != &other) { - if(h_) - h_.destroy(); + destroy_frame(); h_ = std::exchange(other.h_, nullptr); } return *this; } private: + // Set while a parent is suspended awaiting this quitter. + bool lent_ = false; + explicit quitter(std::coroutine_handle h) : h_(h) { } + + void destroy_frame() noexcept + { + if(!h_) + return; + if(lent_ && !h_.done()) + { + // Only the unwind of an abandoned chain destroys a frame + // that is still awaited. + auto* owner = h_.promise().environment()->owner; + if(!owner || !owner->abandoned()) + std::terminate(); // LCOV_EXCL_LINE reached only in a death test's child process + } + std::exchange(h_, nullptr).destroy(); + lent_ = false; + } }; } // namespace capy diff --git a/include/boost/capy/task.hpp b/include/boost/capy/task.hpp index 1a4ee909b..aabf6c792 100644 --- a/include/boost/capy/task.hpp +++ b/include/boost/capy/task.hpp @@ -364,11 +364,18 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE */ std::coroutine_handle h_; - /// Destroy the task and its coroutine frame if owned. + /** Destroy the task and its coroutine frame if owned. + + @par Preconditions + The frame is not lent: no parent is suspended awaiting it, + unless its chain is being unwound after an @ref abandon. + Destroying a lent frame otherwise calls `std::terminate`; end + a running chain by requesting stop, or by destroying its + execution context. + */ ~task() { - if(h_) - h_.destroy(); + destroy_frame(); } /** Report whether the awaited task is already complete. @@ -399,6 +406,7 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE [[nodiscard]] auto await_resume() requires detail::is_io_result_v { + lent_ = false; if(h_.promise().has_ep_) std::rethrow_exception(h_.promise().ep_); return std::move(*h_.promise().result_); @@ -407,6 +415,7 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE auto await_resume() requires (! detail::is_io_result_v) { + lent_ = false; if(h_.promise().has_ep_) std::rethrow_exception(h_.promise().ep_); if constexpr (! std::is_void_v) @@ -434,18 +443,28 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE { h_.promise().set_continuation(cont); h_.promise().set_environment(env); + lent_ = true; return h_; } /** Return the coroutine handle. @note Do not call `destroy()` on the returned handle while the - task is being awaited. The task's lifetime is normally managed - by `run_async`, `run`, or the awaiting parent. Manually - destroying a suspended task that another coroutine is awaiting - produces undefined behavior. For cooperative cancellation, use + task is being awaited: the awaiting frame still holds it and + destroys it again, which has undefined behavior. A holder that + gives the coroutine up passes its continuation to @ref abandon + instead. (Destroying or + move-assigning over the task object itself while its frame is + lent is a different operation, and calls `std::terminate`.) + The task's lifetime is normally managed by `run_async`, `run`, + or the awaiting parent. For cooperative cancellation, use `std::stop_token`. + @note The operation, queue or primitive holding a suspended + coroutine gives it up with @ref abandon, through the + continuation it keeps, never by destroying a handle. + `handle()` belongs to the owning task object. + @return The coroutine handle. */ std::coroutine_handle handle() const noexcept @@ -461,8 +480,8 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE @note The caller may call `destroy()` on the released handle only when the task has not started or has fully completed. - Destroying a suspended task that is being awaited produces - undefined behavior. + Destroying the released handle of a task that is being + awaited is not supported and has undefined behavior. @par Postconditions `handle()` returns a null handle. Callers needing the @@ -505,6 +524,10 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE If this task already owns a coroutine frame, that frame is destroyed first. Self-assignment is a no-op. + @par Preconditions + The frame being replaced is not lent: no parent is suspended + awaiting it. Replacing a lent frame calls `std::terminate`. + @par Postconditions `other` is empty and must not be awaited. @@ -516,18 +539,36 @@ struct [[nodiscard]] BOOST_CAPY_CORO_AWAIT_ELIDABLE { if(this != &other) { - if(h_) - h_.destroy(); + destroy_frame(); h_ = std::exchange(other.h_, nullptr); } return *this; } private: + // Set while a parent is suspended awaiting this task. + bool lent_ = false; + explicit task(std::coroutine_handle h) : h_(h) { } + + void destroy_frame() noexcept + { + if(!h_) + return; + if(lent_ && !h_.done()) + { + // Only the unwind of an abandoned chain destroys a frame + // that is still awaited. + auto* owner = h_.promise().environment()->owner; + if(!owner || !owner->abandoned()) + std::terminate(); // LCOV_EXCL_LINE reached only in a death test's child process + } + std::exchange(h_, nullptr).destroy(); + lent_ = false; + } }; } // namespace capy diff --git a/include/boost/capy/test/run_blocking.hpp b/include/boost/capy/test/run_blocking.hpp index 6c3a1299c..9571dc554 100644 --- a/include/boost/capy/test/run_blocking.hpp +++ b/include/boost/capy/test/run_blocking.hpp @@ -14,11 +14,13 @@ #include #include #include +#include #include #include #include #include +#include #include #include @@ -32,7 +34,8 @@ class blocking_context; This executor is used internally by @ref run_blocking to execute coroutine tasks on the calling thread. Work submitted - via `dispatch()` is returned for symmetric transfer. Work + via `dispatch()` from within the context's event loop is + returned for symmetric transfer; from elsewhere it is queued. Work submitted via `post()` is enqueued and processed by the @ref blocking_context event loop. @@ -80,14 +83,17 @@ struct BOOST_CAPY_DECL blocking_executor /// Called when work completes (no-op). void on_work_finished() const noexcept; - /** Dispatch work for immediate inline execution. + /** Dispatch work, running it inline when already in the loop. - Returns the handle for symmetric transfer. The caller - resumes the coroutine via the returned handle. + Called from within the context's event loop, returns the + handle for the caller to resume by symmetric transfer. + Called from anywhere else, including while the context is + shutting down, queues the continuation as @ref post does. @param c The continuation to execute. - @return `c.h` for symmetric transfer. + @return `c.h` for symmetric transfer, or + `std::noop_coroutine()` if the continuation was queued. */ std::coroutine_handle<> dispatch(continuation& c) const; @@ -118,10 +124,13 @@ struct BOOST_CAPY_DECL blocking_executor This context is created internally by @ref run_blocking. Users do not interact with it directly. + Coroutines still queued when the context is destroyed are + passed to @ref abandon rather than resumed. + @par Thread Safety The event loop runs on the thread that calls `run()`. - `signal_done()` and `enqueue()` are safe to call from - any thread. + `signal_done()`, `signal_abandoned()`, and `enqueue()` are safe + to call from any thread. @see blocking_executor, run_blocking */ @@ -170,6 +179,14 @@ class BOOST_CAPY_DECL blocking_context void signal_done(std::exception_ptr ep) noexcept; + /** Signal that the task was abandoned without a result. + + Wakes the event loop so that @ref run throws + `std::system_error` with `error::canceled`. + */ + void + signal_abandoned() noexcept; + /** Run the event loop until done. Blocks the calling thread, processing posted @@ -184,12 +201,15 @@ class BOOST_CAPY_DECL blocking_context void run(); - /** Enqueue a coroutine handle for processing. + /** Enqueue a continuation for processing. + + A copy is queued, so the owner it carries reaches @ref abandon + if the context is destroyed first. - @param h The coroutine handle to enqueue. + @param c The continuation to enqueue. */ void - enqueue(std::coroutine_handle<> h); + enqueue(continuation const& c); }; /** Wrapper that signals completion after invoking the handler. @@ -216,6 +236,43 @@ struct blocking_handler_wrapper /// The success and error handlers to forward to. detail::handler_pair handlers_; + /** Destroy the wrapper. + + A wrapper destroyed without being invoked belongs to an + abandoned chain, which delivers no result. The context is + signalled with `error::canceled`, so its loop neither waits + forever nor reports success. + */ + ~blocking_handler_wrapper() + { + if(ctx_) + ctx_->signal_abandoned(); + } + + /** Construct from a context and handlers. + + @param ctx The context to signal. + + @param handlers The handlers to forward to. + */ + blocking_handler_wrapper( + blocking_context* ctx, + detail::handler_pair handlers) + : ctx_(ctx) + , handlers_(std::move(handlers)) + { + } + + /** Construct by moving, leaving `other` unable to signal. + + @param other The wrapper to move from. + */ + blocking_handler_wrapper(blocking_handler_wrapper&& other) noexcept + : ctx_(std::exchange(other.ctx_, nullptr)) + , handlers_(std::move(other.handlers_)) + { + } + /** Invoke the handler with a non-void result. @param v The result value to forward to the handler. @@ -229,10 +286,11 @@ struct blocking_handler_wrapper } catch(...) { - ctx_->signal_done(std::current_exception()); + std::exchange(ctx_, nullptr)->signal_done( + std::current_exception()); return; } - ctx_->signal_done(); + std::exchange(ctx_, nullptr)->signal_done(); } /** Invoke the handler for a void result. */ @@ -244,10 +302,11 @@ struct blocking_handler_wrapper } catch(...) { - ctx_->signal_done(std::current_exception()); + std::exchange(ctx_, nullptr)->signal_done( + std::current_exception()); return; } - ctx_->signal_done(); + std::exchange(ctx_, nullptr)->signal_done(); } /** Invoke the handler with an exception. @@ -262,10 +321,11 @@ struct blocking_handler_wrapper } catch(...) { - ctx_->signal_done(std::current_exception()); + std::exchange(ctx_, nullptr)->signal_done( + std::current_exception()); return; } - ctx_->signal_done(); + std::exchange(ctx_, nullptr)->signal_done(); } }; @@ -359,6 +419,9 @@ class [[nodiscard]] run_blocking_wrapper event loop, starts the task via `run_async`, then pumps the loop until the task completes or throws. + @throws std::system_error with `error::canceled` if the task + is abandoned before it completes. + @tparam Task The IoRunnable type. @param t The task to execute. diff --git a/include/boost/capy/test/stream.hpp b/include/boost/capy/test/stream.hpp index 3b7f5d5c4..069b3997e 100644 --- a/include/boost/capy/test/stream.hpp +++ b/include/boost/capy/test/stream.hpp @@ -75,7 +75,9 @@ class stream { std::string buf; std::size_t max_read_size = std::size_t(-1); - continuation pending_cont_; + // The parked read's own continuation: an executor may queue it + // in place, so it is never copied or changed once posted. + continuation* pending_cont_ = nullptr; executor_ref pending_ex; // Points at the suspended reader's claim flag (owned by the // read awaitable). Lets a peer wake coordinate with a stop @@ -100,17 +102,16 @@ class stream // stop callback; the loser of the race skips the post. static void wake(half& side) { - if(! side.pending_cont_.h) + auto* c = std::exchange(side.pending_cont_, nullptr); + if(! c) return; - if(! side.pending_claimed || - ! side.pending_claimed->exchange( - true, std::memory_order_acq_rel)) + auto ex = std::exchange(side.pending_ex, {}); + auto* claimed = std::exchange(side.pending_claimed, nullptr); + if(! claimed || + ! claimed->exchange(true, std::memory_order_acq_rel)) { - side.pending_ex.post(side.pending_cont_); + ex.post(*c); } - side.pending_cont_.h = {}; - side.pending_ex = {}; - side.pending_claimed = nullptr; } // Set closed and resume any suspended readers @@ -315,7 +316,7 @@ class stream // peer wake does not dereference a freed claim flag. if(side_ && side_->pending_claimed == &claimed_) { - side_->pending_cont_.h = {}; + side_->pending_cont_ = nullptr; side_->pending_ex = {}; side_->pending_claimed = nullptr; } @@ -335,23 +336,25 @@ class stream !side.buf.empty(); } - std::coroutine_handle<> await_suspend( + bool await_suspend( std::coroutine_handle<> h, io_env const* env) noexcept { - // Park the continuation, then register the stop callback. - // If stop is already requested, the callback fires inline - // during construction: it claims the resume and posts the - // continuation through the executor (never a symmetric - // self-transfer, which would leak this frame under - // run_async). The parked read is then resumed with - // error::canceled by the run loop. + // A callback registered after stop was requested would + // fire inline and post this coroutine, which could then be + // resumed and finished before the registration returns. + if(env->stop_token.stop_requested()) + { + canceled_ = true; + return false; + } auto& side = self_->state_->sides[ self_->index_]; cont_.h = h; + cont_.owner = env->owner; ex_ = env->executor; side_ = &side; - side.pending_cont_.h = h; + side.pending_cont_ = &cont_; side.pending_ex = env->executor; side.pending_claimed = &claimed_; @@ -359,7 +362,7 @@ class stream env->stop_token, cancel_fn{this}); stop_cb_active_ = true; - return std::noop_coroutine(); + return true; } [[nodiscard]] io_result @@ -380,7 +383,7 @@ class stream // untouched; unlink if a peer wake has not already. if(side_ && side_->pending_claimed == &claimed_) { - side_->pending_cont_.h = {}; + side_->pending_cont_ = nullptr; side_->pending_ex = {}; side_->pending_claimed = nullptr; } diff --git a/include/boost/capy/when_all.hpp b/include/boost/capy/when_all.hpp index a649fd4c3..9f44fbf6e 100644 --- a/include/boost/capy/when_all.hpp +++ b/include/boost/capy/when_all.hpp @@ -17,6 +17,9 @@ #include #include #include +#include +#include +#include #include #include #include @@ -81,7 +84,14 @@ struct when_all_core struct stop_callback_fn { std::stop_source* source_; - void operator()() const { source_->request_stop(); } + void operator()() const + { + // Stopping can lose the last branch and destroy the join, + // with its source, before the request returns; the copy + // keeps the stop state alive until then. + std::stop_source keep = *source_; + keep.request_stop(); + } }; using stop_callback_t = std::stop_callback; std::optional parent_stop_callback_; @@ -89,6 +99,22 @@ struct when_all_core continuation continuation_; io_env const* caller_env_ = nullptr; + // Set when a branch is abandoned before finishing: the parent is + // then abandoned rather than resumed. + std::atomic lost_{false}; + + /// Drop a share held by a runner that never finished. + void lose_branch() noexcept + { + lost_.store(true, std::memory_order_relaxed); + stop_source_.request_stop(); + if(remaining_count_.fetch_sub(1, std::memory_order_acq_rel) == 1) + { + auto parent = continuation_; + capy::abandon(parent); + } + } + explicit when_all_core(std::size_t count) noexcept : remaining_count_(count) { @@ -209,14 +235,23 @@ struct when_all_homogeneous_state> @tparam StateType The state type (when_all_state or when_all_homogeneous_state). */ template -struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE when_all_runner +struct when_all_runner { struct promise_type - : frame_alloc_mixin + : detail::process_frame_mixin { StateType* state_ = nullptr; std::size_t index_ = 0; io_env env_; + // Owns the branch's segment. + chain_owner segment_; + bool done_ = false; + + ~promise_type() + { + if(!done_ && state_) + state_->core_.lose_branch(); + } when_all_runner get_return_object() noexcept { @@ -239,15 +274,31 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE when_all_runner { auto& core = p_->state_->core_; auto* counter = &core.remaining_count_; + auto* lost = &core.lost_; auto* caller_env = core.caller_env_; auto& cont = core.continuation_; + p_->done_ = true; + // Resumed to unwind its abandoned branch: lost, not + // finished. Stop is requested while this branch + // still holds its share. + if(p_->segment_.abandoned()) + { + lost->store(true, std::memory_order_relaxed); + core.stop_source_.request_stop(); + } h.destroy(); auto remaining = counter->fetch_sub(1, std::memory_order_acq_rel); - if(remaining == 1) - return detail::symmetric_transfer(caller_env->executor.dispatch(cont)); - return detail::symmetric_transfer(std::noop_coroutine()); + if(remaining != 1) + return detail::symmetric_transfer(std::noop_coroutine()); + if(lost->load(std::memory_order_relaxed)) + { + auto parent = cont; + capy::abandon(parent); + return detail::symmetric_transfer(std::noop_coroutine()); + } + return detail::symmetric_transfer(caller_env->executor.dispatch(cont)); } void await_resume() const noexcept {} // LCOV_EXCL_LINE final_suspend awaiter, never resumed }; @@ -328,9 +379,16 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE when_all_runner */ template when_all_runner -make_when_all_homogeneous_runner(Awaitable inner, StateType* state, std::size_t index) +make_when_all_homogeneous_runner(Awaitable* inner, StateType* state, std::size_t index) { - auto result = co_await std::move(inner); + // Taken from the join's storage only once running, into a local: + // a runner destroyed unstarted owns no branch, and one destroyed + // unfinished destroys it before its promise reports the loss. + Awaitable a(std::move(*inner)); + auto r = co_await abandonable{a}; + if(!r) + co_return; + auto& result = *r; if(std::get<0>(result)) { @@ -352,11 +410,17 @@ make_when_all_homogeneous_runner(Awaitable inner, StateType* state, std::size_t /** Create a runner for io_result children that requests stop on ec. */ template when_all_runner> -make_when_all_io_runner(Awaitable inner, when_all_state* state) +make_when_all_io_runner(Awaitable* inner, when_all_state* state) { - auto result = co_await std::move(inner); - auto ec = std::get<0>(result); - std::get(state->results_).set(std::move(result)); + // Taken from the join's storage only once running, into a local: + // a runner destroyed unstarted owns no branch, and one destroyed + // unfinished destroys it before its promise reports the loss. + Awaitable a(std::move(*inner)); + auto r = co_await abandonable{a}; + if(!r) + co_return; + auto ec = std::get<0>(*r); + std::get(state->results_).set(std::move(*r)); if(ec) { @@ -392,6 +456,7 @@ class when_all_io_launcher std::coroutine_handle<> continuation, io_env const* caller_env) { state_->core_.continuation_.h = continuation; + state_->core_.continuation_.owner = caller_env->owner; state_->core_.caller_env_ = caller_env; if(caller_env->stop_token.stop_possible()) @@ -425,12 +490,14 @@ class when_all_io_launcher void launch_one(executor_ref caller_ex, std::stop_token token) { auto runner = make_when_all_io_runner( - std::move(std::get(*awaitables_)), state_); + &std::get(*awaitables_), state_); auto h = runner.release(); h.promise().state_ = state_; + h.promise().segment_.bind(h, caller_ex); h.promise().env_ = io_env{caller_ex, token, - state_->core_.caller_env_->frame_allocator}; + state_->core_.caller_env_->frame_allocator, + &h.promise().segment_}; state_->runner_handles_[I].h = std::coroutine_handle<>{h}; state_->core_.caller_env_->executor.post(state_->runner_handles_[I]); @@ -492,6 +559,7 @@ class when_all_homogeneous_launcher std::coroutine_handle<> await_suspend(std::coroutine_handle<> continuation, io_env const* caller_env) { state_->core_.continuation_.h = continuation; + state_->core_.continuation_.owner = caller_env->owner; state_->core_.caller_env_ = caller_env; if(caller_env->stop_token.stop_possible()) @@ -511,12 +579,14 @@ class when_all_homogeneous_launcher for(auto&& a : *range_) { auto runner = make_when_all_homogeneous_runner( - std::move(a), state_, index); + &a, state_, index); auto h = runner.release(); h.promise().state_ = state_; h.promise().index_ = index; - h.promise().env_ = io_env{caller_env->executor, token, caller_env->frame_allocator}; + h.promise().segment_.bind(h, caller_env->executor); + h.promise().env_ = io_env{caller_env->executor, token, + caller_env->frame_allocator, &h.promise().segment_}; state_->runner_handles_[index].h = std::coroutine_handle<>{h}; ++index; @@ -617,6 +687,12 @@ class when_all_homogeneous_launcher If a child throws, the first child exception is rethrown after all children complete (exception beats error_code). + @note If a branch's coroutine is abandoned before it finishes, for + example by teardown of its execution context, `when_all` + requests stop on the remaining branches, and the awaiting + parent is abandoned rather than resumed once the last branch + has finished or been abandoned. + @par Example @par !example example_1 @@ -638,13 +714,13 @@ template using Awaitable = std::ranges::range_value_t; using PayloadT = detail::io_result_payload_t< awaitable_result_t>; - using OwnedRange = std::remove_cvref_t; + using OwnedRange = detail::owned_range_t; auto count = std::ranges::size(awaitables); if(count == 0) throw std::invalid_argument("when_all requires at least one awaitable"); - OwnedRange owned_awaitables = std::forward(awaitables); + OwnedRange owned_awaitables = detail::own_range(std::forward(awaitables)); detail::when_all_homogeneous_state state(count); @@ -730,6 +806,12 @@ template If a child throws, the first child exception is rethrown after all children complete (exception beats error_code). + @note If a branch's coroutine is abandoned before it finishes, for + example by teardown of its execution context, `when_all` + requests stop on the remaining branches, and the awaiting + parent is abandoned rather than resumed once the last branch + has finished or been abandoned. + @par Example @par !example example_2 @@ -745,13 +827,13 @@ template std::tuple<>> [[nodiscard]] auto when_all(R&& awaitables) -> task> { - using OwnedRange = std::remove_cvref_t; + using OwnedRange = detail::owned_range_t; auto count = std::ranges::size(awaitables); if(count == 0) throw std::invalid_argument("when_all requires at least one awaitable"); - OwnedRange owned_awaitables = std::forward(awaitables); + OwnedRange owned_awaitables = detail::own_range(std::forward(awaitables)); detail::when_all_homogeneous_state> state(count); @@ -827,6 +909,12 @@ template If a child throws, the first child exception is rethrown after all children complete (exception beats error_code). + @note If a branch's coroutine is abandoned before it finishes, for + example by teardown of its execution context, `when_all` + requests stop on the remaining branches, and the awaiting + parent is abandoned rather than resumed once the last branch + has finished or been abandoned. + @param awaitables One or more awaitables each returning io_result. diff --git a/include/boost/capy/when_any.hpp b/include/boost/capy/when_any.hpp index 74e2ed2c8..9518aa653 100644 --- a/include/boost/capy/when_any.hpp +++ b/include/boost/capy/when_any.hpp @@ -18,6 +18,9 @@ #include #include #include +#include +#include +#include #include #include #include @@ -139,7 +142,14 @@ struct when_any_core struct stop_callback_fn { std::stop_source* source_; - void operator()() const noexcept { source_->request_stop(); } + void operator()() const noexcept + { + // Stopping can lose the last branch and destroy the join, + // with its source, before the request returns; the copy + // keeps the stop state alive until then. + std::stop_source keep = *source_; + keep.request_stop(); + } }; using stop_callback_t = std::stop_callback; std::optional parent_stop_callback_; @@ -147,6 +157,22 @@ struct when_any_core continuation continuation_; io_env const* caller_env_ = nullptr; + // Set when a branch is abandoned before finishing: the parent is + // then abandoned rather than resumed. + std::atomic lost_{false}; + + /// Drop a share held by a runner that never finished. + void lose_branch() noexcept + { + lost_.store(true, std::memory_order_relaxed); + stop_source_.request_stop(); + if(remaining_count_.fetch_sub(1, std::memory_order_acq_rel) == 1) + { + auto parent = continuation_; + capy::abandon(parent); + } + } + // Placed last to avoid padding (1-byte atomic followed by 8-byte aligned members) std::atomic has_winner_{false}; @@ -223,14 +249,23 @@ struct when_any_io_state // Wrapper coroutine for io_result-aware when_any children. // unhandled_exception records the exception but does NOT claim winner status. template -struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE when_any_io_runner +struct when_any_io_runner { struct promise_type - : frame_alloc_mixin + : detail::process_frame_mixin { StateType* state_ = nullptr; std::size_t index_ = 0; io_env env_; + // Owns the branch's segment. + chain_owner segment_; + bool done_ = false; + + ~promise_type() + { + if(!done_ && state_) + state_->core_.lose_branch(); + } when_any_io_runner get_return_object() noexcept { @@ -250,15 +285,31 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE when_any_io_runner { auto& core = p_->state_->core_; auto* counter = &core.remaining_count_; + auto* lost = &core.lost_; auto* caller_env = core.caller_env_; auto& cont = core.continuation_; + p_->done_ = true; + // Resumed to unwind its abandoned branch: lost, not + // finished. Stop is requested while this branch + // still holds its share. + if(p_->segment_.abandoned()) + { + lost->store(true, std::memory_order_relaxed); + core.stop_source_.request_stop(); + } h.destroy(); auto remaining = counter->fetch_sub(1, std::memory_order_acq_rel); - if(remaining == 1) - return detail::symmetric_transfer(caller_env->executor.dispatch(cont)); - return detail::symmetric_transfer(std::noop_coroutine()); + if(remaining != 1) + return detail::symmetric_transfer(std::noop_coroutine()); + if(lost->load(std::memory_order_relaxed)) + { + auto parent = cont; + capy::abandon(parent); + return detail::symmetric_transfer(std::noop_coroutine()); + } + return detail::symmetric_transfer(caller_env->executor.dispatch(cont)); } void await_resume() const noexcept {} // LCOV_EXCL_LINE final_suspend awaiter, never resumed }; @@ -334,9 +385,16 @@ struct BOOST_CAPY_CORO_DESTROY_WHEN_COMPLETE when_any_io_runner // Runner coroutine: only tries to win when the child returns !ec. template when_any_io_runner -make_when_any_io_runner(Awaitable inner, StateType* state) +make_when_any_io_runner(Awaitable* inner, StateType* state) { - auto result = co_await std::move(inner); + // Taken from the join's storage only once running, into a local: + // a runner destroyed unstarted owns no branch, and one destroyed + // unfinished destroys it before its promise reports the loss. + Awaitable a(std::move(*inner)); + auto r = co_await abandonable{a}; + if(!r) + co_return; + auto& result = *r; if(!std::get<0>(result)) { @@ -390,6 +448,7 @@ class when_any_io_launcher std::coroutine_handle<> continuation, io_env const* caller_env) { state_->core_.continuation_.h = continuation; + state_->core_.continuation_.owner = caller_env->owner; state_->core_.caller_env_ = caller_env; if(caller_env->stop_token.stop_possible()) @@ -423,13 +482,15 @@ class when_any_io_launcher void launch_one(executor_ref caller_ex, std::stop_token token) { auto runner = make_when_any_io_runner( - std::move(std::get(*tasks_)), state_); + &std::get(*tasks_), state_); auto h = runner.release(); h.promise().state_ = state_; h.promise().index_ = I; + h.promise().segment_.bind(h, caller_ex); h.promise().env_ = io_env{caller_ex, token, - state_->core_.caller_env_->frame_allocator}; + state_->core_.caller_env_->frame_allocator, + &h.promise().segment_}; state_->runner_handles_[I].h = std::coroutine_handle<>{h}; caller_ex.post(state_->runner_handles_[I]); @@ -511,9 +572,16 @@ struct when_any_io_homogeneous_state> template when_any_io_runner make_when_any_io_homogeneous_runner( - Awaitable inner, StateType* state, std::size_t index) + Awaitable* inner, StateType* state, std::size_t index) { - auto result = co_await std::move(inner); + // Taken from the join's storage only once running, into a local: + // a runner destroyed unstarted owns no branch, and one destroyed + // unfinished destroys it before its promise reports the loss. + Awaitable a(std::move(*inner)); + auto r = co_await abandonable{a}; + if(!r) + co_return; + auto& result = *r; if(!std::get<0>(result)) { @@ -570,6 +638,7 @@ class when_any_io_homogeneous_launcher std::coroutine_handle<> continuation, io_env const* caller_env) { state_->core_.continuation_.h = continuation; + state_->core_.continuation_.owner = caller_env->owner; state_->core_.caller_env_ = caller_env; if(caller_env->stop_token.stop_possible()) @@ -589,13 +658,14 @@ class when_any_io_homogeneous_launcher for(auto&& a : *range_) { auto runner = make_when_any_io_homogeneous_runner( - std::move(a), state_, index); + &a, state_, index); auto h = runner.release(); h.promise().state_ = state_; h.promise().index_ = index; + h.promise().segment_.bind(h, caller_env->executor); h.promise().env_ = io_env{caller_env->executor, token, - caller_env->frame_allocator}; + caller_env->frame_allocator, &h.promise().segment_}; state_->runner_handles_[index].h = std::coroutine_handle<>{h}; ++index; @@ -694,6 +764,12 @@ class when_any_io_homogeneous_launcher fail and the reported failure is an exception, that child's exception is rethrown (which child is unspecified). + @note If a branch's coroutine is abandoned before it finishes, for + example by teardown of its execution context, `when_any` + requests stop on the remaining branches, and the awaiting + parent is abandoned rather than resumed once the last branch + has finished or been abandoned. + @par Example @par !example example_1 @@ -718,13 +794,13 @@ template awaitable_result_t>; using result_type = std::variant>; - using OwnedRange = std::remove_cvref_t; + using OwnedRange = detail::owned_range_t; auto count = std::ranges::size(awaitables); if(count == 0) throw std::invalid_argument("when_any requires at least one awaitable"); - OwnedRange owned_awaitables = std::forward(awaitables); + OwnedRange owned_awaitables = detail::own_range(std::forward(awaitables)); detail::when_any_io_homogeneous_state state(count); @@ -813,6 +889,12 @@ template If all children fail and the reported failure is an exception, that child's exception is rethrown (which child is unspecified). + @note If a branch's coroutine is abandoned before it finishes, for + example by teardown of its execution context, `when_any` + requests stop on the remaining branches, and the awaiting + parent is abandoned rather than resumed once the last branch + has finished or been abandoned. + @par Example @par !example example_2 @@ -829,14 +911,14 @@ template [[nodiscard]] auto when_any(R&& awaitables) -> task> { - using OwnedRange = std::remove_cvref_t; + using OwnedRange = detail::owned_range_t; using result_type = std::variant; auto count = std::ranges::size(awaitables); if(count == 0) throw std::invalid_argument("when_any requires at least one awaitable"); - OwnedRange owned_awaitables = std::forward(awaitables); + OwnedRange owned_awaitables = detail::own_range(std::forward(awaitables)); detail::when_any_io_homogeneous_state> state(count); @@ -938,6 +1020,12 @@ template benign error (e.g. @c cond::canceled) count as a win, wrap the child to translate the error into success. See the Concurrent Composition tutorial. + + @note If a branch's coroutine is abandoned before it finishes, for + example by teardown of its execution context, `when_any` + requests stop on the remaining branches, and the awaiting + parent is abandoned rather than resumed once the last branch + has finished or been abandoned. */ template requires (sizeof...(As) > 0) diff --git a/src/ex/detail/strand_queue.hpp b/src/ex/detail/strand_queue.hpp index d3dba3cbf..d34106844 100644 --- a/src/ex/detail/strand_queue.hpp +++ b/src/ex/detail/strand_queue.hpp @@ -13,8 +13,11 @@ #include #include +#include #include +#include + namespace boost { namespace capy { namespace detail { @@ -63,6 +66,25 @@ class strand_queue tail_ = &c; } + /// Return the first pending continuation, or null if empty. + continuation* + front() const noexcept + { + return head_; + } + + /** Remove the first pending continuation. + + @pre The queue is not empty. + */ + void + pop_front() noexcept + { + head_ = static_cast(head_->reserved); + if(!head_) + tail_ = nullptr; + } + /** Batch of taken items for thread-safe dispatch. */ struct taken_batch { @@ -108,6 +130,28 @@ class strand_queue } batch.tail = nullptr; } + + /** Abandon each coroutine in a taken batch without resuming it. + + Used when the strand can no longer run them. Advances past + each node first, since the node may live in a frame the + abandon destroys. + + @param batch The batch to abandon. + */ + static + void + abandon_batch(taken_batch& batch) noexcept + { + while(batch.head) + { + continuation* c = batch.head; + batch.head = static_cast(c->reserved); + if(c->h) + capy::abandon(*c); + } + batch.tail = nullptr; + } }; } // namespace detail diff --git a/src/ex/detail/strand_service.cpp b/src/ex/detail/strand_service.cpp index 17f9b4698..96dc9d78e 100644 --- a/src/ex/detail/strand_service.cpp +++ b/src/ex/detail/strand_service.cpp @@ -10,8 +10,10 @@ #include "src/ex/detail/strand_impl.hpp" #include #include +#include #include #include +#include #include namespace boost { @@ -42,6 +44,7 @@ class strand_service_impl : public strand_service std::shared_ptr mutexes_[num_mutexes]; intrusive_list impl_list_; std::atomic invoker_frame_cache_{nullptr}; + bool shutdown_ = false; explicit strand_service_impl(execution_context&) @@ -54,6 +57,13 @@ class strand_service_impl : public strand_service auto new_impl = std::make_shared(); std::lock_guard lock(mutex_); + if(shutdown_) + { + // Born closed, like the strands shutdown() closed: it + // borrows nothing from this service. + new_impl->locked_ = true; + return new_impl; + } std::size_t s = salt_++; std::size_t idx = reinterpret_cast(new_impl.get()); @@ -70,17 +80,72 @@ class strand_service_impl : public strand_service return new_impl; } - static bool + // Returns the service when the caller must post an invoker. + static strand_service_impl* enqueue(strand_impl& impl, continuation& c) + { + // A closed strand never locks: the mutex it borrowed may have + // gone with its service. + if(impl.service_.load(std::memory_order_acquire)) + { + std::lock_guard lock(*impl.mutex_); + if(auto* svc = impl.service_.load(std::memory_order_relaxed)) + { + impl.pending_.push(c); + if(!impl.locked_) + { + impl.locked_ = true; + return svc; + } + return nullptr; + } + } + // Shut down: nothing will run it, so abandon it as its holder. + capy::abandon(c); + return nullptr; + } + + enum class withdrawal + { + alone, + followed, + taken + }; + + // Takes back a continuation whose invoker could not be posted. + // `taken` means a shutdown took it first and abandoned it. + static withdrawal + withdraw(strand_impl& impl, continuation& c) noexcept { std::lock_guard lock(*impl.mutex_); - impl.pending_.push(c); - if(!impl.locked_) + if(!impl.service_.load(std::memory_order_relaxed)) + return withdrawal::taken; + // Nothing takes from the queue while it waits for an invoker, + // so `c` is still first. + BOOST_CAPY_ASSERT(impl.pending_.front() == &c); + std::ignore = c; + impl.pending_.pop_front(); + // Posts queued behind `c` returned trusting its invoker, so + // the strand stays locked until one runs them. + if(!impl.pending_.empty()) + return withdrawal::followed; + impl.locked_ = false; + return withdrawal::alone; + } + + // Gives up queued work that no invoker can be posted to run. + static void + abandon_pending(strand_impl& impl) noexcept + { + strand_queue::taken_batch batch; { - impl.locked_ = true; - return true; + std::lock_guard lock(*impl.mutex_); + if(!impl.service_.load(std::memory_order_relaxed)) + return; + batch = impl.pending_.take_all(); + impl.locked_ = false; } - return false; + strand_queue::abandon_batch(batch); } static void @@ -120,19 +185,38 @@ class strand_service_impl : public strand_service // Defined below; needs strand_invoker complete. static void - post_invoker(std::shared_ptr impl, executor_ref ex); + post_invoker( + strand_service_impl* svc, + std::shared_ptr impl, + executor_ref ex); protected: void shutdown() override { - std::lock_guard lock(mutex_); - while(auto* p = impl_list_.pop_front()) + strand_queue::taken_batch doomed; { - std::lock_guard impl_lock(*p->mutex_); - p->locked_ = true; - p->service_.store(nullptr, std::memory_order_release); + std::lock_guard lock(mutex_); + shutdown_ = true; + while(auto* p = impl_list_.pop_front()) + { + std::lock_guard impl_lock(*p->mutex_); + p->locked_ = true; + p->service_.store(nullptr, std::memory_order_release); + auto b = p->pending_.take_all(); + if(b.head) + { + if(doomed.tail) + doomed.tail->reserved = b.head; + else + doomed.head = b.head; + doomed.tail = b.tail; + } + } } + // Outside the mutexes: a destroyed frame may drop the last + // strand handle and re-enter the service. + strand_queue::abandon_batch(doomed); void* fp = invoker_frame_cache_.exchange( kCacheClosed, std::memory_order_acq_rel); @@ -160,18 +244,27 @@ struct strand_invoker // posting to the inner executor. continuation self_; + // The service is the one enqueue() saw under the strand's lock; + // the impl's own pointer may be cleared by a concurrent shutdown. void* operator new( std::size_t n, - std::shared_ptr const& impl) + strand_service_impl* svc, + std::shared_ptr const&) { - auto* svc = impl->service_.load(std::memory_order_acquire); constexpr auto A = alignof(strand_service_impl*); std::size_t padded = (n + A - 1) & ~(A - 1); std::size_t total = padded + sizeof(strand_service_impl*); - void* p = svc->invoker_frame_cache_.exchange( - nullptr, std::memory_order_acquire); + // Never takes the closed marker, which would let a later + // delete park a frame that nothing frees. + void* p = svc->invoker_frame_cache_.load( + std::memory_order_acquire); + while(p && p != kCacheClosed && + !svc->invoker_frame_cache_.compare_exchange_weak( + p, nullptr, std::memory_order_acquire)) + { + } if(!p || p == kCacheClosed) p = ::operator new(total); @@ -214,7 +307,7 @@ struct strand_invoker // strand drop. static strand_invoker -make_invoker(std::shared_ptr impl) +make_invoker(strand_service_impl*, std::shared_ptr impl) { auto* p = impl.get(); for(;;) @@ -231,21 +324,69 @@ make_invoker(std::shared_ptr impl) void strand_service_impl::post_invoker( + strand_service_impl* svc, std::shared_ptr impl, executor_ref ex) { - auto invoker = make_invoker(std::move(impl)); + auto invoker = make_invoker(svc, std::move(impl)); auto& self = invoker.h_.promise().self_; self.h = invoker.h_; - ex.post(self); + try + { + ex.post(self); + } + catch(...) + { + invoker.h_.destroy(); + throw; + } +} + +// A post that throws leaves the caller holding `c`, unless a +// shutdown already took and abandoned it. +static +void +post_or_withdraw( + strand_service_impl* svc, + std::shared_ptr const& impl, + executor_ref ex, + continuation& c) +{ + try + { + strand_service_impl::post_invoker(svc, impl, ex); + } + catch(...) + { + using withdrawal = strand_service_impl::withdrawal; + auto const w = strand_service_impl::withdraw(*impl, c); + if(w == withdrawal::taken) + return; + if(w == withdrawal::followed) + { + try + { + strand_service_impl::post_invoker(svc, impl, ex); + } + catch(...) + { + strand_service_impl::abandon_pending(*impl); + } + } + throw; + } } strand_impl::~strand_impl() { - auto* svc = service_.load(std::memory_order_acquire); - if(!svc) return; - std::lock_guard lock(svc->mutex_); - svc->impl_list_.remove(this); + if(auto* svc = service_.load(std::memory_order_acquire)) + { + std::lock_guard lock(svc->mutex_); + svc->impl_list_.remove(this); + } + // Nothing can run these any more: give them up. + auto batch = pending_.take_all(); + strand_queue::abandon_batch(batch); } strand_service:: @@ -274,8 +415,8 @@ dispatch( if(running_in_this_thread(*impl)) return c.h; - if(strand_service_impl::enqueue(*impl, c)) - strand_service_impl::post_invoker(impl, ex); + if(auto* svc = strand_service_impl::enqueue(*impl, c)) + post_or_withdraw(svc, impl, ex, c); return std::noop_coroutine(); } @@ -286,8 +427,8 @@ post( executor_ref ex, continuation& c) { - if(strand_service_impl::enqueue(*impl, c)) - strand_service_impl::post_invoker(impl, ex); + if(auto* svc = strand_service_impl::enqueue(*impl, c)) + post_or_withdraw(svc, impl, ex, c); } strand_service& diff --git a/src/ex/execution_context.cpp b/src/ex/execution_context.cpp index bc79345b4..2a67b8fe8 100644 --- a/src/ex/execution_context.cpp +++ b/src/ex/execution_context.cpp @@ -47,13 +47,12 @@ void execution_context:: destroy() noexcept { - service* p = head_; - head_ = nullptr; - while(p) + // One at a time, so a destructor can still find the services not + // yet destroyed, and one it creates is destroyed in turn. + while(service* p = head_) { - service* next = p->next_; + head_ = p->next_; delete p; - p = next; } } @@ -98,6 +97,13 @@ use_service_impl(factory& f) sp->next_ = head_; head_ = sp; + if(shutdown_) + { + // Created after the context shut down: it starts shut down, + // like the services that already were. + lock.unlock(); + sp->shutdown(); + } return *sp; } @@ -116,7 +122,7 @@ make_service_impl(factory& f) // Unlocked to allow nested service creation from constructor service* p = f.create(*this); - std::lock_guard lock(mutex_); + std::unique_lock lock(mutex_); if(find_impl(f.t0)) { delete p; // LCOV_EXCL_LINE concurrent same-type registration race @@ -141,6 +147,13 @@ make_service_impl(factory& f) p->next_ = head_; head_ = p; + if(shutdown_) + { + // Created after the context shut down: it starts shut down, + // like the services that already were. + lock.unlock(); + p->shutdown(); + } return *p; } diff --git a/src/ex/frame_allocator.cpp b/src/ex/frame_allocator.cpp new file mode 100644 index 000000000..6132c1a1f --- /dev/null +++ b/src/ex/frame_allocator.cpp @@ -0,0 +1,30 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +#include + +#if (defined(BOOST_CAPY_DYN_LINK) || defined(BOOST_ALL_DYN_LINK)) && \ + !defined(BOOST_CAPY_STATIC_LINK) + +namespace boost { +namespace capy { +namespace detail { + +std::pmr::memory_resource*& +current_frame_allocator_ref() noexcept +{ + static thread_local std::pmr::memory_resource* mr = nullptr; + return mr; +} + +} // namespace detail +} // namespace capy +} // namespace boost + +#endif diff --git a/src/ex/recycling_memory_resource.cpp b/src/ex/recycling_memory_resource.cpp index e6f0db05a..2d188adfd 100644 --- a/src/ex/recycling_memory_resource.cpp +++ b/src/ex/recycling_memory_resource.cpp @@ -10,6 +10,7 @@ #include #include +#include namespace boost { namespace capy { @@ -20,7 +21,34 @@ namespace capy { // independent of any instance lifetime. recycling_memory_resource::~recycling_memory_resource() = default; -void +#if (defined(BOOST_CAPY_DYN_LINK) || defined(BOOST_ALL_DYN_LINK)) && \ + !defined(BOOST_CAPY_STATIC_LINK) + +recycling_memory_resource::pool& +recycling_memory_resource::local() noexcept +{ + static thread_local pool p; + return p; +} + +bool& +recycling_memory_resource::thread_armed() noexcept +{ + static thread_local bool armed = false; + return armed; +} + +#endif + +namespace { + +// Set once this thread's pool was drained at exit. Another thread_local +// destroyed later can still free frames; they must not be cached. +thread_local bool pool_drained = false; + +} // namespace + +bool recycling_memory_resource::arm_thread_cleanup() noexcept { struct janitor @@ -34,10 +62,15 @@ recycling_memory_resource::arm_thread_cleanup() noexcept for(auto& b : lp.buckets) while(b.count > 0) ::operator delete(b.pop()); + pool_drained = true; + thread_armed() = false; } }; + if(pool_drained) + return false; static thread_local janitor j; - (void)j; + std::ignore = j; + return true; } recycling_memory_resource::pool& @@ -81,10 +114,13 @@ void* recycling_memory_resource::allocate_slow( std::size_t rounded, std::size_t idx) { - arm_thread_cleanup(); + // Once this thread's pool was drained at exit, take one block and + // leave the pool empty: nothing would release a refill. + bool const cache = arm_thread_cleanup(); { std::lock_guard lock(global_mutex()); - if(auto* p = global().buckets[idx].pop(local().buckets[idx])) + auto& b = global().buckets[idx]; + if(auto* p = cache ? b.pop(local().buckets[idx]) : b.pop()) return p; } return ::operator new(rounded); diff --git a/src/ex/thread_pool.cpp b/src/ex/thread_pool.cpp index 7327c83b8..68cdf1df7 100644 --- a/src/ex/thread_pool.cpp +++ b/src/ex/thread_pool.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -30,23 +31,23 @@ directly. Worker threads wait on a condition_variable until work is available or stop is requested. - Threads are started lazily on first post() via std::call_once to avoid - spawning threads for pools that are constructed but never used. Each - thread is named with a configurable prefix plus index for debugger - visibility. + Threads are started lazily on first post(), under mutex_, to avoid + spawning threads for pools that are constructed but never used. A + stopped pool starts none. Each thread is named + with a configurable prefix plus index for debugger visibility. Work tracking: on_work_started/on_work_finished maintain the atomic - outstanding_work_ counter. on_work_started is lock-free; the worker - that drives the count to zero takes mutex_ and re-reads the count - before deciding to stop, so the count and the stop decision stay - consistent even if work is started in between. join() blocks until - this counter reaches zero, then signals workers to stop and joins - threads. + outstanding_work_ counter. on_work_started is lock-free, and so is + on_work_finished while the count stays above one. The release that + may take it to zero is made under mutex_, so join() can't see zero, + return and let the pool be destroyed while that call still uses it. + join() blocks until this counter reaches zero, then signals workers + to stop and joins threads. Two shutdown paths: - join(): waits for outstanding work to drain, then stops workers. - - stop(): immediately signals workers to exit; queued work is abandoned. - - Destructor: stop() then join() (abandon + wait for threads). + - stop(): immediately signals workers to exit; queued work stays queued. + - Destructor: stop() then join(), then queued work is abandoned. */ namespace boost { @@ -103,7 +104,7 @@ class thread_pool::impl bool joined_{false}; std::size_t num_threads_; char thread_name_prefix_[13]{}; // 12 chars max + null terminator - std::once_flag start_flag_; + std::atomic started_{false}; public: ~impl() = default; @@ -114,19 +115,28 @@ class thread_pool::impl return current_.get() == this; } - // Destroy abandoned coroutine frames. Must be called - // before execution_context::shutdown()/destroy() so - // that suspended-frame destructors touching services - // (e.g. cancelling registrations) run while those - // services are still valid. - void + // Abandon queued coroutines. Called after the services shut down, + // since their shutdown can post more, and again after they are + // destroyed, for anything their destructors post. Each is taken + // under the lock, as other threads may still post, and abandoned + // outside it, as abandoning can post. Returns whether there was + // anything to abandon. + bool drain_abandoned() noexcept { - while(auto* c = pop()) + bool any = false; + for(;;) { - auto h = c->h; - if(h && h != std::noop_coroutine()) - h.destroy(); + continuation* c; + { + std::lock_guard lock(mutex_); + c = pop(); + } + if(!c) + return any; + any = true; + if(c->h && c->h != std::noop_coroutine()) + capy::abandon(*c); } } @@ -145,14 +155,25 @@ class thread_pool::impl void post(continuation& c) { - ensure_started(); + if(!started_.load(std::memory_order_acquire)) { - std::lock_guard lock(mutex_); - push(&c); - // Under the lock so the pool cannot drain, join, and - // destroy the condition variable mid-signal. - work_cv_.notify_one(); + { + std::lock_guard lock(mutex_); + if(stop_) + { + // Stopped before any thread started: nothing will + // run it, and the destructor's drain abandons it. + push(&c); + return; + } + } + ensure_started(); } + std::lock_guard lock(mutex_); + push(&c); + // Under the lock so the pool cannot drain, join, and + // destroy the condition variable mid-signal. + work_cv_.notify_one(); } void @@ -164,20 +185,22 @@ class thread_pool::impl void on_work_finished() noexcept { - if(outstanding_work_.fetch_sub( - 1, std::memory_order_acq_rel) == 1) + auto n = outstanding_work_.load(std::memory_order_relaxed); + while(n > 1) { - // fetch_sub's result can be stale: a concurrent - // on_work_started() may raise the count before we take the - // lock, so re-read it here rather than trust the decrement. - std::lock_guard lock(mutex_); - if(outstanding_work_.load( - std::memory_order_acquire) == 0 && joined_ && !stop_) - { - stop_ = true; - done_cv_.notify_all(); - work_cv_.notify_all(); - } + if(outstanding_work_.compare_exchange_weak( + n, n - 1, std::memory_order_acq_rel)) + return; + } + std::lock_guard lock(mutex_); + outstanding_work_.fetch_sub(1, std::memory_order_acq_rel); + // Re-read: a concurrent on_work_started() may have raised it. + if(outstanding_work_.load( + std::memory_order_acquire) == 0 && joined_ && !stop_) + { + stop_ = true; + done_cv_.notify_all(); + work_cv_.notify_all(); } } @@ -212,10 +235,10 @@ class thread_pool::impl void stop() noexcept { - { - std::lock_guard lock(mutex_); - stop_ = true; - } + // Under the lock: the join() this releases may return and let + // the pool be destroyed as soon as the lock is free. + std::lock_guard lock(mutex_); + stop_ = true; work_cv_.notify_all(); done_cv_.notify_all(); } @@ -224,11 +247,16 @@ class thread_pool::impl void ensure_started() { - std::call_once(start_flag_, [this]{ - threads_.reserve(num_threads_); - for(std::size_t i = 0; i < num_threads_; ++i) - threads_.emplace_back([this, i]{ run(i); }); - }); + // Under the lock, so join() either sees every thread started + // here or makes this start none. A joined pool still starts them: + // join() may be waiting on work that needs them. + std::lock_guard lock(mutex_); + if(started_.load(std::memory_order_relaxed) || stop_) + return; + threads_.reserve(num_threads_); + for(std::size_t i = 0; i < num_threads_; ++i) + threads_.emplace_back([this, i]{ run(i); }); + started_.store(true, std::memory_order_release); } void @@ -273,9 +301,15 @@ thread_pool:: { impl_->stop(); impl_->join(); - impl_->drain_abandoned(); shutdown(); - destroy(); + impl_->drain_abandoned(); + // A frame the drain unwinds can create a service, whose destructor + // can post again: repeat until nothing is left of either. + do + { + destroy(); + } + while(impl_->drain_abandoned()); delete impl_; } diff --git a/src/test/run_blocking.cpp b/src/test/run_blocking.cpp index 143dc5a26..fda50d140 100644 --- a/src/test/run_blocking.cpp +++ b/src/test/run_blocking.cpp @@ -9,22 +9,34 @@ #include +#include #include #include #include #include +#include +#include namespace boost { namespace capy { namespace test { +namespace { + +// The context whose run() is on this thread's stack. Only there is it +// safe to resume dispatched work inline. +thread_local blocking_context const* running_ = nullptr; + +} // namespace + struct blocking_context::impl { std::mutex mtx; std::condition_variable cv; - std::queue> queue; + std::queue queue; std::exception_ptr ep; bool done = false; + bool abandoned = false; }; blocking_context::blocking_context() @@ -34,6 +46,36 @@ blocking_context::blocking_context() blocking_context::~blocking_context() { + // Nothing will run these now; abandon them as their holder. The + // services may post while they shut down or are destroyed, so the + // queue outlives both. Each is taken under the lock, as other + // threads may still post, and abandoned outside it. + auto drain = [this] + { + bool any = false; + for(;;) + { + continuation c; + { + std::lock_guard lock(impl_->mtx); + if(impl_->queue.empty()) + return any; + c = impl_->queue.front(); + impl_->queue.pop(); + } + any = true; + capy::abandon(c); + } + }; + shutdown(); + drain(); + // A frame the drain unwinds can create a service, whose destructor + // can post again: repeat until nothing is left of either. + do + { + destroy(); + } + while(drain()); delete impl_; } @@ -61,12 +103,36 @@ blocking_context::signal_done( impl_->cv.notify_one(); } +void +blocking_context::signal_abandoned() noexcept +{ + std::lock_guard lock(impl_->mtx); + impl_->abandoned = true; + impl_->done = true; + impl_->cv.notify_one(); +} + void blocking_context::run() { + struct scoped_running + { + blocking_context const* prev_; + + explicit scoped_running(blocking_context const* ctx) noexcept + : prev_(std::exchange(running_, ctx)) + { + } + + ~scoped_running() + { + running_ = prev_; + } + } guard(this); + for(;;) { - std::coroutine_handle<> h; + continuation c; { std::unique_lock lock(impl_->mtx); impl_->cv.wait(lock, [&] { @@ -74,21 +140,23 @@ blocking_context::run() }); if(impl_->done && impl_->queue.empty()) break; - h = impl_->queue.front(); + c = impl_->queue.front(); impl_->queue.pop(); } - safe_resume(h); + safe_resume(c.h); } if(impl_->ep) std::rethrow_exception(impl_->ep); + if(impl_->abandoned) + throw std::system_error(error::canceled, "task abandoned"); } void blocking_context::enqueue( - std::coroutine_handle<> h) + continuation const& c) { std::lock_guard lock(impl_->mtx); - impl_->queue.push(h); + impl_->queue.push(c); impl_->cv.notify_one(); } @@ -121,14 +189,17 @@ std::coroutine_handle<> blocking_executor::dispatch( continuation& c) const { - return c.h; + if(running_ == ctx_) + return c.h; + ctx_->enqueue(c); + return std::noop_coroutine(); } void blocking_executor::post( continuation& c) const { - ctx_->enqueue(c.h); + ctx_->enqueue(c); } } // namespace test diff --git a/test/doc/snippets/4h_allocators.cpp b/test/doc/snippets/4h_allocators.cpp index 5555bef99..3f8baa73c 100644 --- a/test/doc/snippets/4h_allocators.cpp +++ b/test/doc/snippets/4h_allocators.cpp @@ -126,8 +126,9 @@ capy::task<> process(item const&) co_return; } -// Bound to a blocking (inline) executor in the test so every frame is -// released before the stack buffer backing the resource goes away. +// Bound to a blocking executor in the test, and process_batch is called +// from inside its loop, so every frame runs inline and is released before +// the stack buffer backing the resource goes away. capy::any_executor executor; // tag::batch_allocator[] @@ -141,10 +142,16 @@ void process_batch(std::vector const& items) { capy::run_async(executor, &resource)(process(item)); } - // All frames deallocated when resource goes out of scope + // The resource is released here, so every task must have finished } // end::batch_allocator[] +capy::task<> batch_inline(std::vector const& items) +{ + process_batch(items); + co_return; +} + // tag::recycling_observe_resource[] // A memory resource that pools freed blocks by size. When a coroutine // frame is freed, its block is kept; the next frame of the same size @@ -417,7 +424,9 @@ struct allocators_test executor = ctx.get_executor(); std::vector items(3); int const before = items_processed; - process_batch(items); + capy::run_async(executor, [&] { ctx.signal_done(); })( + batch_inline(items)); + ctx.run(); BOOST_TEST(items_processed == before + 3); executor = capy::any_executor(); } diff --git a/test/doc/snippets/4i_lambda_captures.cpp b/test/doc/snippets/4i_lambda_captures.cpp index aba5218eb..f04321c8c 100644 --- a/test/doc/snippets/4i_lambda_captures.cpp +++ b/test/doc/snippets/4i_lambda_captures.cpp @@ -200,6 +200,10 @@ struct lambda_captures_test socket sock; int const before = socket_reads; iife_param::process(sock); + // The snippet's run_async has no handler to signal completion; + // run() still drains what is queued before it sees the signal. + ctx.signal_done(); + ctx.run(); BOOST_TEST(socket_reads == before + 1); executor = capy::any_executor(); } diff --git a/test/doc/snippets/4j_lifetimes.cpp b/test/doc/snippets/4j_lifetimes.cpp new file mode 100644 index 000000000..f9e4acd9d --- /dev/null +++ b/test/doc/snippets/4j_lifetimes.cpp @@ -0,0 +1,220 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +// Compiled fragments shown in pages/4.coroutines/4j.lifetimes.adoc. +// The fragments that demonstrate dangling references are compiled but +// never executed: running them would be undefined behavior. + +#include "../doc_warnings.hpp" + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "test_suite.hpp" + +namespace capy = boost::capy; + +namespace { + +capy::any_executor executor; + +// tag::ref_param_awaited[] +capy::task count_bytes(std::string const& text) +{ + co_return text.size(); +} + +capy::task awaited_at_once() +{ + // The temporary lives until the end of the full expression, + // and the full expression includes the whole co_await. + co_return co_await count_bytes(std::string("hello")); +} +// end::ref_param_awaited[] + +// tag::ref_param_deferred[] +capy::task awaited_later() +{ + auto t = count_bytes(std::string("hello")); // <1> + co_return co_await std::move(t); // <2> +} +// end::ref_param_deferred[] + +// tag::value_param[] +capy::task count_owned_bytes(std::string text) +{ + co_return text.size(); +} + +capy::task awaited_later_by_value() +{ + auto t = count_owned_bytes(std::string("hello")); + co_return co_await std::move(t); +} +// end::value_param[] + +void start_dangling() +{ + // tag::run_async_temporary[] + capy::run_async(executor)(count_bytes(std::string("hello"))); // <1> + // end::run_async_temporary[] +} + +void start_owned() +{ + // tag::run_async_owned[] + capy::run_async(executor)(count_owned_bytes(std::string("hello"))); + // end::run_async_owned[] +} + +// tag::unwind[] +struct slot_guard +{ + int& in_use; + + explicit slot_guard(int& n) noexcept + : in_use(n) + { + ++in_use; + } + + ~slot_guard() + { + --in_use; // <1> + } +}; + +capy::task<> wait_for_work(capy::async_event& ready, int& in_use) +{ + slot_guard guard(in_use); + auto [ec] = co_await ready.wait(); + if(ec) + co_return; + // <2> +} +// end::unwind[] + +bool stopped_through_own_code = false; + +capy::task<> wait_until_stopped(capy::async_event& never_set) +{ + auto [ec] = co_await never_set.wait(); + stopped_through_own_code = (ec == capy::cond::canceled); +} + +capy::task<> noop() +{ + co_return; +} + +struct lifetimes_test +{ + void + testAwaitedAtOnce() + { + std::size_t n = 0; + capy::test::run_blocking([&](std::size_t v) { n = v; })( + awaited_at_once()); + BOOST_TEST(n == 5); + } + + void + testValueParam() + { + std::size_t n = 0; + capy::test::run_blocking([&](std::size_t v) { n = v; })( + awaited_later_by_value()); + BOOST_TEST(n == 5); + } + + void + testGuardOnCompletion() + { + int in_use = 0; + capy::async_event ready; + ready.set(); + capy::test::run_blocking()(wait_for_work(ready, in_use)); + BOOST_TEST(in_use == 0); + } + + // The pool is stopped while the coroutine waits, so destroying the + // event posts it to a queue that never runs, and destroying the + // pool abandons it. + void + testGuardOnUnwind() + { + int in_use = 0; + bool completed = false; + { + capy::thread_pool pool(1); + capy::async_event ready; + capy::run_async(pool.get_executor(), + [&] { completed = true; }, + [&](std::exception_ptr) { completed = true; })( + wait_for_work(ready, in_use)); + + // The one worker runs in order, so this runs only once the + // first coroutine is suspended in wait(). + std::latch suspended(1); + capy::run_async(pool.get_executor(), + [&] { suspended.count_down(); })(noop()); + suspended.wait(); + BOOST_TEST(in_use == 1); + pool.stop(); + pool.join(); + } + BOOST_TEST(in_use == 0); + BOOST_TEST(! completed); + } + + void + testShutdownOrder() + { + stopped_through_own_code = false; + capy::async_event never_set; + // tag::shutdown_order[] + capy::thread_pool pool(1); + std::stop_source stop; + capy::run_async(pool.get_executor(), stop.get_token())( + wait_until_stopped(never_set)); + + // Later, to shut down: + stop.request_stop(); // <1> + pool.join(); // <2> + // <3> + // end::shutdown_order[] + BOOST_TEST(stopped_through_own_code); + } + + void + run() + { + testAwaitedAtOnce(); + testValueParam(); + testGuardOnCompletion(); + testGuardOnUnwind(); + testShutdownOrder(); + } +}; + +} // namespace + +TEST_SUITE(lifetimes_test, "boost.capy.doc.4j_lifetimes"); diff --git a/test/doc/snippets/9k_executor.cpp b/test/doc/snippets/9k_executor.cpp index 6d1cbfcb5..3576d9ce7 100644 --- a/test/doc/snippets/9k_executor.cpp +++ b/test/doc/snippets/9k_executor.cpp @@ -14,6 +14,7 @@ #include #include +#include #include #include #include @@ -66,6 +67,7 @@ struct continuation { std::coroutine_handle<> h; void* reserved = nullptr; + capy::chain_owner* owner = nullptr; }; // end::continuation_struct[] diff --git a/test/unit/custom_task.hpp b/test/unit/custom_task.hpp index a2d85fe6d..e37849140 100644 --- a/test/unit/custom_task.hpp +++ b/test/unit/custom_task.hpp @@ -82,17 +82,23 @@ struct custom_task explicit custom_task(std::coroutine_handle h) : h_(h) {} - ~custom_task() { if(h_) h_.destroy(); } + ~custom_task() { destroy(); } custom_task(custom_task const&) = delete; custom_task& operator=(custom_task const&) = delete; custom_task(custom_task&& o) noexcept : h_(std::exchange(o.h_, nullptr)) {} custom_task& operator=(custom_task&& o) noexcept { - if(this != &o) { if(h_) h_.destroy(); h_ = std::exchange(o.h_, nullptr); } + if(this != &o) { destroy(); h_ = std::exchange(o.h_, nullptr); } return *this; } + void destroy() noexcept + { + if(h_) + std::exchange(h_, nullptr).destroy(); + } + bool await_ready() const noexcept { return false; } auto await_resume() diff --git a/test/unit/death_test.hpp b/test/unit/death_test.hpp new file mode 100644 index 000000000..4ca21be7c --- /dev/null +++ b/test/unit/death_test.hpp @@ -0,0 +1,95 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +#ifndef BOOST_CAPY_TEST_DEATH_TEST_HPP +#define BOOST_CAPY_TEST_DEATH_TEST_HPP + +#if !defined(_WIN32) + +#include + +#include +#include +#include + +#include +#include +#include + +namespace boost { +namespace capy { + +/// The exit status of a child that reached `std::terminate`. +inline constexpr int terminated_status = 77; + +/** Run a function in a child process and return its wait status. + + The child exits 0, or 1 if `fn` returns false. A child that calls + `std::terminate` exits with @ref terminated_status, so an assertion + or other abort is not mistaken for it. A child still running after + a minute is killed, since one forked from a multithreaded process + can deadlock. +*/ +template +int +run_in_child(F fn) +{ + pid_t pid = ::fork(); + if(pid == 0) + { + // Keep diagnostics out of the suite's output. + int fd = ::open("/dev/null", O_WRONLY); + if(fd >= 0) + ::dup2(fd, 2); + ::alarm(60); + std::set_terminate([] { ::_exit(terminated_status); }); + bool ok = true; + if constexpr(std::is_same_v) + ok = fn(); + else + fn(); + ::_exit(ok ? 0 : 1); + } + int status = 0; + ::waitpid(pid, &status, 0); + return status; +} + +/// Return true when `fn` calls `std::terminate`. +template +bool +terminates(F fn) +{ + int status = run_in_child(fn); + return WIFEXITED(status) && WEXITSTATUS(status) == terminated_status; +} + +/// Return true when `fn` returns, and returns true if it returns a bool. +template +bool +exits_cleanly(F fn) +{ + int status = run_in_child(fn); + return WIFEXITED(status) && WEXITSTATUS(status) == 0; +} + +/// Start a task by hand, as an awaiting parent would, and leave it lent. +template +void +lend(Task& t, io_env& env) +{ + t.await_suspend(std::noop_coroutine(), &env).resume(); +} + +} // capy +} // boost + +#endif + +#endif diff --git a/test/unit/ex/abandon.cpp b/test/unit/ex/abandon.cpp new file mode 100644 index 000000000..7cc74424a --- /dev/null +++ b/test/unit/ex/abandon.cpp @@ -0,0 +1,632 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +// Test that header file is self-contained. +#include + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "test_helpers.hpp" + +namespace boost { +namespace capy { + +namespace { + +// Parks the awaiting coroutine without the environment protocol, +// taking the owner from its promise instead. +struct plain_park +{ + parked* slot; + + bool await_ready() const noexcept { return false; } + + template + void await_suspend(std::coroutine_handle h) noexcept + { + slot->h = h; + slot->owner = h.promise().environment()->owner; + } + + void await_resume() const noexcept {} +}; + +/** A coroutine type whose promise deallocates its frame itself. + + Its allocation goes through the frame allocator in scope, like + `frame_alloc_mixin`, but its `operator delete` calls nothing else. +*/ +struct own_delete_task +{ + struct promise_type + : io_awaitable_promise_base + { + static void* operator new(std::size_t size) + { + auto* mr = get_current_frame_allocator(); + if(!mr) + mr = std::pmr::get_default_resource(); + auto total = size + sizeof(std::pmr::memory_resource*); + void* raw = mr->allocate(total, alignof(std::max_align_t)); + std::memcpy( + static_cast(raw) + size, &mr, + sizeof(std::pmr::memory_resource*)); + return raw; + } + + static void operator delete(void* p, std::size_t size) noexcept + { + std::pmr::memory_resource* mr; + std::memcpy( + &mr, static_cast(p) + size, + sizeof(std::pmr::memory_resource*)); + mr->deallocate( + p, size + sizeof(std::pmr::memory_resource*), + alignof(std::max_align_t)); + } + + own_delete_task get_return_object() + { + return own_delete_task{ + std::coroutine_handle::from_promise(*this)}; + } + + std::suspend_always initial_suspend() noexcept { return {}; } + + auto final_suspend() noexcept + { + struct awaiter + { + promise_type* p_; + bool await_ready() const noexcept { return false; } + std::coroutine_handle<> await_suspend( + std::coroutine_handle<>) const noexcept + { + return p_->continuation(); + } + void await_resume() const noexcept {} + }; + return awaiter{this}; + } + + void return_void() noexcept {} + void unhandled_exception() noexcept { std::terminate(); } + }; + + std::coroutine_handle h_; + + explicit own_delete_task(std::coroutine_handle h) + : h_(h) + { + } + + own_delete_task(own_delete_task&& o) noexcept + : h_(std::exchange(o.h_, nullptr)) + { + } + + ~own_delete_task() + { + if(h_) + { + auto const* env = h_.promise().environment(); + auto* owner = env ? env->owner : nullptr; + if(lent_ && !h_.done() && !(owner && owner->abandoned())) + std::terminate(); + h_.destroy(); + } + } + + bool await_ready() const noexcept { return false; } + + std::coroutine_handle<> await_suspend( + std::coroutine_handle<> cont, io_env const* env) + { + h_.promise().set_continuation(cont); + h_.promise().set_environment(env); + lent_ = true; + return h_; + } + + void await_resume() noexcept + { + lent_ = false; + } + +private: + bool lent_ = false; +}; + +// Destroys a parked coroutine of another chain when the frame that +// owns it as a by-value parameter is destroyed. +struct holder_param +{ + parked* h; + + explicit holder_param(parked& slot) noexcept + : h(&slot) + { + } + + holder_param(holder_param&& o) noexcept + : h(std::exchange(o.h, nullptr)) + { + } + + ~holder_param() + { + if(h && *h) + { + auto c = std::exchange(*h, parked{}); + capy::abandon(c); + } + } +}; + +/** Parks any number of coroutines; destroying it destroys them all. + + Plays a holder whose teardown runs inside an outer one. +*/ +struct parking_lot +{ + std::vector hs; + + ~parking_lot() + { + for(auto& c : hs) + capy::abandon(c); + } + + struct awaiter + { + parking_lot* lot; + + bool await_ready() const noexcept { return false; } + + std::coroutine_handle<> + await_suspend(std::coroutine_handle<> h, io_env const* env) + { + lot->hs.push_back(continuation{h, nullptr, env->owner}); + return std::noop_coroutine(); + } + + void await_resume() const noexcept {} + }; + + awaiter park() noexcept { return awaiter{this}; } +}; + +// Records the thread that destroys the frame holding it. +struct thread_mark +{ + std::thread::id* where; + + explicit thread_mark(std::thread::id& w) noexcept + : where(&w) + { + } + + thread_mark(thread_mark&& o) noexcept + : where(std::exchange(o.where, nullptr)) + { + } + + ~thread_mark() + { + if(where) + *where = std::this_thread::get_id(); + } +}; + +// Parks the awaiting coroutine and signals that it did. +struct signal_park +{ + parked* slot; + std::binary_semaphore* done; + + bool await_ready() const noexcept { return false; } + + std::coroutine_handle<> + await_suspend(std::coroutine_handle<> h, io_env const* env) noexcept + { + slot->h = h; + slot->owner = env->owner; + done->release(); + return std::noop_coroutine(); + } + + void await_resume() const noexcept {} +}; + +// Counts posts and runs nothing. +struct counting_executor +{ + int* posts; + + bool operator==(counting_executor const& o) const noexcept + { + return posts == o.posts; + } + test_io_context& context() const noexcept + { + return default_test_io_context(); + } + void on_work_started() const noexcept {} + void on_work_finished() const noexcept {} + std::coroutine_handle<> dispatch(continuation& c) const + { + post(c); + return std::noop_coroutine(); + } + void post(continuation&) const + { + ++*posts; + } +}; + +} // namespace + +struct abandon_test +{ + // A holder owned by a frame that the teardown destroys: every + // parked coroutine's parent is collected, with no limit on how many. + void testCascadeDestroysThousandWaiters() + { + constexpr int count = 1000; + int inner_down = 0, outer_down = 0, server_down = 0; + parked slot; + std::queue q; + queuing_executor ex(q); + auto inner = [](parking_lot& l, destroy_count_t) -> task + { + co_await l.park(); + }; + auto outer = [&](parking_lot& l, destroy_count_t) -> task + { + co_await inner(l, destroy_count_t(inner_down)); + }; + auto leaf = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto server = [&](destroy_count_t) -> task + { + parking_lot lot; + for(int i = 0; i < count; ++i) + run_async(ex)(outer(lot, destroy_count_t(outer_down))); + co_await leaf(&slot); + }; + run_async(ex)(server(destroy_count_t(server_down))); + pump(q); + BOOST_TEST(slot); + BOOST_TEST_EQ(inner_down, 0); + + capy::abandon(slot); + pump(q); + BOOST_TEST_EQ(inner_down, count); + BOOST_TEST_EQ(outer_down, count); + BOOST_TEST_EQ(server_down, 1); + } + + // A teardown started from a thread_local destructor, as its thread + // exits, destroys the chain and frees every frame. + void testDestroyFromThreadLocalDestructor() + { + alloc_counts st; + int child_down = 0, parent_down = 0; + parked slot; + { + auto child = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await child(&slot, destroy_count_t(child_down)); + }; + int n = 0; + test_executor ex(n); + run_async(ex, counting_allocator(st))( + parent(destroy_count_t(parent_down))); + BOOST_TEST(slot); + std::thread([&slot] { + struct exit_holder + { + parked* h; + ~exit_holder() { capy::abandon(*h); } + }; + thread_local exit_holder holder{&slot}; + }).join(); + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.balanced()); + } + + // A thread_local holder made before its thread first allocates a + // frame outlives that thread's frame cache; the frames its teardown + // frees at exit are released, not cached where nothing frees them. + // A leak here shows only under LeakSanitizer. + void testDestroyAtThreadExitAfterCacheDrained() + { + int child_down = 0, parent_down = 0; + std::thread([&] { + struct exit_holder + { + parked h; + ~exit_holder() + { + if(h) + capy::abandon(h); + } + }; + thread_local exit_holder holder; + auto child = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await child(&holder.h, destroy_count_t(child_down)); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + BOOST_TEST(holder.h); + }).join(); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + } + + // A teardown started while an exception unwinds the stack destroys + // the chain as usual. + void testDestroyDuringUnwinding() + { + int child_down = 0, parent_down = 0; + parked slot; + auto child = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await child(&slot, destroy_count_t(child_down)); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + BOOST_TEST(slot); + + struct unwind_holder + { + parked* h; + ~unwind_holder() { capy::abandon(*h); } + }; + try + { + unwind_holder holder{&slot}; + throw std::runtime_error("unwind"); + } + catch(std::runtime_error const&) + { + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + } + + // A promise that frees its own frame through the allocator held by + // the run_async trampoline needs nothing from capy: the trampoline + // is destroyed only after that frame is freed. + void testOwnOperatorDeleteNeedsNoHook() + { + alloc_counts st; + int child_down = 0, parent_down = 0; + parked slot; + { + auto child = [](parked* s, destroy_count_t) -> own_delete_task + { + co_await plain_park{s}; + }; + auto parent = [&](parked* s, destroy_count_t) + -> task + { + co_await child(s, destroy_count_t(child_down)); + }; + int n = 0; + test_executor ex(n); + run_async(ex, counting_allocator(st))( + parent(&slot, destroy_count_t(parent_down))); + BOOST_TEST(slot); + + capy::abandon(slot); + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // A teardown started while a frame is between its promise and its + // deallocation leaves that frame's parent alone: the parent's chain + // owns the allocator the frame is about to be freed through. + void testNestedDestroyKeepsOuterParent() + { + alloc_counts st; + int parent_down = 0, other_down = 0, other_parent_down = 0; + parked slot, other; + auto other_leaf = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto other_parent = [&](destroy_count_t) -> task + { + co_await other_leaf(&other, destroy_count_t(other_down)); + }; + auto leaf = [](holder_param, parked* s) + -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await leaf(holder_param(other), &slot); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(other_parent(destroy_count_t(other_parent_down))); + run_async(ex, counting_allocator(st))( + parent(destroy_count_t(parent_down))); + BOOST_TEST(slot); + BOOST_TEST(other); + + capy::abandon(slot); + BOOST_TEST_EQ(other_down, 1); + BOOST_TEST_EQ(other_parent_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // A chain is unwound on its owner's executor, not on the thread of + // the holder that gives it up. + void testUnwindRunsOnOwnersExecutor() + { + std::thread::id child_at, root_at; + std::thread::id pool_thread; + parked slot; + std::binary_semaphore done{0}; + thread_pool pool(1); + auto child = [&](thread_mark) -> task + { + co_await signal_park{&slot, &done}; + }; + auto root = [&](thread_mark) -> task + { + pool_thread = std::this_thread::get_id(); + co_await child(thread_mark(child_at)); + }; + run_async(pool.get_executor())(root(thread_mark(root_at))); + // Bounds a hang, not a timing assertion. + if(!BOOST_TEST(done.try_acquire_for(std::chrono::seconds(30)))) + return; + + capy::abandon(slot); + pool.join(); + BOOST_TEST(child_at == pool_thread); + BOOST_TEST(root_at == pool_thread); + BOOST_TEST(pool_thread != std::this_thread::get_id()); + } + + // Across run on another executor, the inner segment is unwound on + // that executor and the parent then on the caller's. + void testUnwindAcrossRunBoundary() + { + std::thread::id child_at, parent_at; + std::thread::id inner_thread, caller_thread; + parked slot; + std::binary_semaphore done{0}; + thread_pool caller(1); + thread_pool inner(1); + auto child = [&](thread_mark) -> task + { + inner_thread = std::this_thread::get_id(); + co_await signal_park{&slot, &done}; + }; + auto parent = [&](thread_mark) -> task + { + caller_thread = std::this_thread::get_id(); + co_await capy::run(inner.get_executor())( + child(thread_mark(child_at))); + }; + run_async(caller.get_executor())(parent(thread_mark(parent_at))); + // Bounds a hang, not a timing assertion. + if(!BOOST_TEST(done.try_acquire_for(std::chrono::seconds(30)))) + return; + + capy::abandon(slot); + // The launch holds work on the caller until its unwind is done, + // which follows the inner one. + caller.join(); + inner.join(); + BOOST_TEST(child_at == inner_thread); + BOOST_TEST(parent_at == caller_thread); + BOOST_TEST(inner_thread != caller_thread); + } + + // A segment is unwound once however many times it is abandoned. + void testSecondAbandonDoesNothing() + { + int posts = 0; + counting_executor ex{&posts}; + chain_owner owner; + owner.bind(std::noop_coroutine(), ex); + owner.abandon(); + owner.abandon(); + BOOST_TEST(owner.abandoned()); + BOOST_TEST_EQ(posts, 1); + } + + // A continuation with no owner is destroyed directly. + void testAbandonWithoutOwnerDestroys() + { + int down = 0; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto t = probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + capy::abandon(c); + BOOST_TEST_EQ(down, 1); + } + + void + run() + { + testCascadeDestroysThousandWaiters(); + testDestroyFromThreadLocalDestructor(); + testDestroyDuringUnwinding(); + testDestroyAtThreadExitAfterCacheDrained(); + testOwnOperatorDeleteNeedsNoHook(); + testNestedDestroyKeepsOuterParent(); + testUnwindRunsOnOwnersExecutor(); + testUnwindAcrossRunBoundary(); + testSecondAbandonDoesNothing(); + testAbandonWithoutOwnerDestroys(); + } +}; + +TEST_SUITE( + abandon_test, + "boost.capy.ex.abandon"); + +} // capy +} // boost diff --git a/test/unit/ex/async_event.cpp b/test/unit/ex/async_event.cpp index ee04532a7..61faf5fb5 100644 --- a/test/unit/ex/async_event.cpp +++ b/test/unit/ex/async_event.cpp @@ -12,6 +12,8 @@ #include #include +#include +#include #include #include #include @@ -21,9 +23,12 @@ #include #include +#include +#include #include #include #include +#include #include #include "test_helpers.hpp" @@ -74,7 +79,7 @@ struct async_event_test testSingleWaiter() { async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); bool resumed = false; @@ -97,7 +102,7 @@ struct async_event_test while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -110,7 +115,7 @@ struct async_event_test testMultipleWaiters() { async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); int count = 0; @@ -148,7 +153,7 @@ struct async_event_test while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -190,7 +195,7 @@ struct async_event_test testSetClearSetSequence() { async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); int count = 0; @@ -213,7 +218,7 @@ struct async_event_test event.set(); while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } BOOST_TEST_EQ(count, 1); @@ -237,7 +242,7 @@ struct async_event_test event.set(); while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } BOOST_TEST_EQ(count, 2); @@ -248,7 +253,7 @@ struct async_event_test testResumeOrder() { async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); std::vector order; @@ -284,7 +289,7 @@ struct async_event_test event.set(); while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -311,7 +316,7 @@ struct async_event_test { // Stop token cancels a waiting coroutine async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; @@ -339,7 +344,7 @@ struct async_event_test while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -355,7 +360,7 @@ struct async_event_test { // Cancel one waiter, set() wakes the rest async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; @@ -398,7 +403,7 @@ struct async_event_test ss.request_stop(); while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } BOOST_TEST(w1_done); @@ -409,7 +414,7 @@ struct async_event_test event.set(); while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } BOOST_TEST(w2_done); @@ -424,7 +429,7 @@ struct async_event_test { // Pre-signaled token on contended event async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; ss.request_stop(); @@ -491,7 +496,7 @@ struct async_event_test // Destroying a coroutine while suspended properly // unlinks the awaiter async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); auto t = [](async_event& ev) -> task { @@ -520,7 +525,7 @@ struct async_event_test // set() claims the waiter first, stop callback // loses the race -- no double resume async_event event; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; @@ -551,7 +556,7 @@ struct async_event_test while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -735,9 +740,196 @@ struct async_event_test BOOST_TEST(finished); } + // A destroyed event can no longer be set, so its waiters resume + // with an error instead of disappearing. + void testDestroyEventCancelsWaiters() + { + int down = 0; + std::error_code ec; + auto ev = std::make_unique(); + auto waiter = [](async_event& e, std::error_code& out, + destroy_count_t) -> task + { + auto [r] = co_await e.wait(); + out = r; + }; + std::queue q; + run_async(queuing_executor(q))(waiter(*ev, ec, destroy_count_t(down))); + pump(q); + ev.reset(); + BOOST_TEST_EQ(down, 0); + BOOST_TEST_EQ(q.size(), 1u); + pump(q); + BOOST_TEST(ec == make_error_code(error::canceled)); + BOOST_TEST_EQ(down, 1); + } + + // An event owned by a frame that an outer teardown destroys posts + // every waiter; each then resumes with an error, or is destroyed + // if its execution context is torn down too. + void testCascadeCancelsManyWaiters() + { + constexpr int count = 64; + int inner_down = 0, outer_down = 0, server_down = 0; + int canceled = 0; + parked slot; + std::queue q; + queuing_executor ex(q); + auto inner = [&](async_event& e, destroy_count_t) -> task + { + auto [r] = co_await e.wait(); + if(r == make_error_code(error::canceled)) + ++canceled; + }; + auto outer = [&](async_event& e, destroy_count_t) -> task + { + co_await inner(e, destroy_count_t(inner_down)); + }; + auto leaf = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto server = [&](destroy_count_t) -> task + { + async_event ev; + for(int i = 0; i < count; ++i) + run_async(ex)(outer(ev, destroy_count_t(outer_down))); + co_await leaf(&slot); + }; + run_async(ex)(server(destroy_count_t(server_down))); + pump(q); + BOOST_TEST(slot); + + capy::abandon(slot); + // Run only the posted unwind; the canceled waiters it posts + // stay queued. + BOOST_TEST_EQ(q.size(), 1u); + auto unwind = q.front(); + q.pop(); + unwind.h.resume(); + BOOST_TEST_EQ(server_down, 1); + BOOST_TEST_EQ(inner_down, 0); + BOOST_TEST_EQ(q.size(), static_cast(count)); + + // Half resume, half are torn down with their context. + for(int i = 0; i < count / 2; ++i) + { + auto c = q.front(); + q.pop(); + c.h.resume(); + } + while(!q.empty()) + { + capy::abandon(q.front()); + q.pop(); + } + BOOST_TEST_EQ(canceled, count / 2); + BOOST_TEST_EQ(inner_down, count); + BOOST_TEST_EQ(outer_down, count); + } + + // Posting a waiter can destroy it on the spot (here, a strand that + // can no longer run it); set() must not touch an event that the + // destroyed frame owned. + void testSetWhenPostDestroysOwner() + { + std::queue q; + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + strand s{qe}; + async_event* ev = nullptr; + int down = 0; + auto waiter = [](async_event*& out, destroy_count_t) -> task + { + async_event e; + out = &e; + std::ignore = co_await e.wait(); + }; + run_async(s, std::allocator{})( + waiter(ev, destroy_count_t(down))); + pump(q); + BOOST_TEST(ev); + ctx->shutdown(); + ev->set(); + BOOST_TEST_EQ(down, 1); + } + + // The same, with set() called from a destructor inside a teardown. + void testSetFromTeardownWhenPostDestroysOwner() + { + std::queue q; + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + strand s{qe}; + async_event* ev = nullptr; + int down = 0; + auto waiter = [](async_event*& out, destroy_count_t) -> task + { + async_event e; + out = &e; + std::ignore = co_await e.wait(); + }; + run_async(s, std::allocator{})( + waiter(ev, destroy_count_t(down))); + pump(q); + + struct set_on_exit + { + async_event** e; + ~set_on_exit() { (*e)->set(); } + }; + parked slot; + std::queue q2; + auto notifier = [](async_event** e, + parked* sl) -> task + { + set_on_exit g{e}; + co_await park_awaitable{sl}; + }; + run_async(queuing_executor(q2), std::allocator{})( + notifier(&ev, &slot)); + pump(q2); + BOOST_TEST(slot); + ctx->shutdown(); + capy::abandon(slot); + pump(q2); + BOOST_TEST_EQ(down, 1); + } + + // A waiter claimed by its stop request belongs to the executor's + // queue: discarding it after the event is gone touches nothing. + void testStopClaimedWaiterOutlivesEvent() + { + auto p = std::make_unique(); + std::queue q; + std::stop_source ss; + int down = 0; + auto waiter = [](async_event& x, destroy_count_t) -> task + { + std::ignore = co_await x.wait(); + }; + run_async(queuing_executor(q), ss.get_token())( + waiter(*p, destroy_count_t(down))); + pump(q); + ss.request_stop(); + BOOST_TEST_EQ(q.size(), 1u); + p.reset(); + capy::abandon(q.front()); + q.pop(); + pump(q); + BOOST_TEST_EQ(down, 1); + } + void run() { + testStopClaimedWaiterOutlivesEvent(); + testSetWhenPostDestroysOwner(); + testSetFromTeardownWhenPostDestroysOwner(); + testDestroyEventCancelsWaiters(); + testCascadeCancelsManyWaiters(); testInitialState(); testSetAndClear(); testWaitOnSetEvent(); diff --git a/test/unit/ex/async_mutex.cpp b/test/unit/ex/async_mutex.cpp index af9c228e6..dad7253a0 100644 --- a/test/unit/ex/async_mutex.cpp +++ b/test/unit/ex/async_mutex.cpp @@ -12,13 +12,18 @@ #include #include +#include #include #include #include +#include +#include #include +#include #include #include +#include // GCC gives false positive -Wmaybe-uninitialized on structured bindings // via the tuple protocol inside coroutine frames. @@ -68,7 +73,7 @@ struct async_mutex_test testContendedLock() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); bool first_done = false; @@ -108,7 +113,7 @@ struct async_mutex_test BOOST_TEST(!second_done); BOOST_TEST(!q.empty()); - q.front().resume(); + q.front().h.resume(); q.pop(); BOOST_TEST(second_done); BOOST_TEST(!cm.is_locked()); @@ -121,7 +126,7 @@ struct async_mutex_test testCancellation() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; @@ -164,7 +169,7 @@ struct async_mutex_test ss.request_stop(); BOOST_TEST(!q.empty()); - q.front().resume(); + q.front().h.resume(); q.pop(); BOOST_TEST(waiter_done); @@ -182,7 +187,7 @@ struct async_mutex_test testScopedLockCancellation() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; @@ -225,7 +230,7 @@ struct async_mutex_test ss.request_stop(); BOOST_TEST(!q.empty()); - q.front().resume(); + q.front().h.resume(); q.pop(); BOOST_TEST(waiter_done); @@ -243,7 +248,7 @@ struct async_mutex_test testDestroyWhileSuspended() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); bool holder_done = false; @@ -286,7 +291,7 @@ struct async_mutex_test testUnlockSkipsCanceled() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss1; std::stop_source ss2; @@ -361,7 +366,7 @@ struct async_mutex_test while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -375,7 +380,7 @@ struct async_mutex_test while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -399,7 +404,7 @@ struct async_mutex_test // remove() must be idempotent so this second unlink is a no-op. // The queuing executor makes the interleaving deterministic. async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss1; @@ -462,7 +467,7 @@ struct async_mutex_test while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -483,7 +488,7 @@ struct async_mutex_test testPreSignaledToken() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; ss.request_stop(); @@ -597,7 +602,7 @@ struct async_mutex_test testPreSignaledScopedLock() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; ss.request_stop(); @@ -651,7 +656,7 @@ struct async_mutex_test testFIFOOrder() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); int order = 0; @@ -709,7 +714,7 @@ struct async_mutex_test cm.unlock(); while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -728,7 +733,7 @@ struct async_mutex_test testCancelMiddleWaiter() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; @@ -796,7 +801,7 @@ struct async_mutex_test ss.request_stop(); while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } BOOST_TEST(w2_done); @@ -805,7 +810,7 @@ struct async_mutex_test cm.unlock(); while(!q.empty()) { - q.front().resume(); + q.front().h.resume(); q.pop(); } @@ -823,7 +828,7 @@ struct async_mutex_test testDestroyMultipleSuspended() { async_mutex cm; - std::queue> q; + std::queue q; queuing_executor ex(q); auto holder = [](async_mutex& cm) -> task { @@ -1009,9 +1014,208 @@ struct async_mutex_test h2.destroy(); } + // A destroyed mutex can no longer grant the lock, so its waiters + // resume with an error instead of disappearing. + void testDestroyMutexCancelsWaiters() + { + int down = 0; + std::error_code ec; + auto m = std::make_unique(); + BOOST_TEST(m->lock().await_ready()); // held by the test + auto waiter = [](async_mutex& mx, std::error_code& out, + destroy_count_t) -> task + { + auto [e] = co_await mx.lock(); + out = e; + }; + std::queue q; + run_async(queuing_executor(q))(waiter(*m, ec, destroy_count_t(down))); + pump(q); + m.reset(); + BOOST_TEST_EQ(down, 0); + BOOST_TEST_EQ(q.size(), 1u); + pump(q); + BOOST_TEST(ec == make_error_code(error::canceled)); + BOOST_TEST_EQ(down, 1); + } + + // A frame owning the mutex is torn down while it holds the lock: + // its guard hands the lock to a waiter, then the mutex dies, then + // the context's teardown destroys that waiter before it resumes. + void testOwnerTornDownWhileHoldingLock() + { + int down = 0; + parked slot; + std::queue q; + queuing_executor ex(q); + int resumed = 0; + auto waiter = [](async_mutex& mx, int& resumed, destroy_count_t) + -> task + { + std::ignore = co_await mx.lock(); + ++resumed; + }; + auto leaf = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto owner = [&]() -> task + { + async_mutex m; + auto [ec, guard] = co_await m.scoped_lock(); + run_async(ex)(waiter(m, resumed, destroy_count_t(down))); + run_async(ex)(waiter(m, resumed, destroy_count_t(down))); + co_await leaf(&slot); + }; + run_async(ex, std::allocator{})(owner()); + pump(q); + BOOST_TEST(slot); + capy::abandon(slot); + while(!q.empty()) + { + capy::abandon(q.front()); + q.pop(); + } + BOOST_TEST_EQ(down, 2); + BOOST_TEST_EQ(resumed, 0); + } + + // The granted waiter's frame owns the mutex, so the post that + // destroys it destroys the mutex while unlock() is still running. + void testUnlockWhenPostDestroysOwner() + { + std::queue q; + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + strand s{qe}; + async_mutex* mp = nullptr; + int down = 0; + auto waiter = [](async_mutex*& out, destroy_count_t) -> task + { + async_mutex m; + out = &m; + std::ignore = co_await m.lock(); + std::ignore = co_await m.lock(); + }; + run_async(s, std::allocator{})( + waiter(mp, destroy_count_t(down))); + pump(q); + BOOST_TEST(mp); + ctx->shutdown(); + mp->unlock(); + BOOST_TEST_EQ(down, 1); + } + + // Each granted waiter destroyed on the spot by its closed strand + // hands the lock on; that must not nest once per waiter. + void testHandoffChainDoesNotRecurse() + { + constexpr int count = 100000; + std::queue q; + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + strand s{qe}; + async_mutex m; + BOOST_TEST(m.lock().await_ready()); + int down = 0; + auto waiter = [](async_mutex& mx, destroy_count_t) -> task + { + std::ignore = co_await mx.lock(); + mx.unlock(); + }; + for(int i = 0; i < count; ++i) + run_async(s)(waiter(m, destroy_count_t(down))); + pump(q); + ctx->shutdown(); + m.unlock(); + BOOST_TEST_EQ(down, count); + BOOST_TEST(!m.is_locked()); + } + + // A waiter the destroyed mutex posted is still destroyed when its + // execution context is torn down instead of run. + void testCanceledWaiterDestroyedAtTeardown() + { + int down = 0; + auto m = std::make_unique(); + BOOST_TEST(m->lock().await_ready()); + auto waiter = [](async_mutex& mx, destroy_count_t) -> task + { + std::ignore = co_await mx.lock(); + }; + std::queue q; + run_async(queuing_executor(q))(waiter(*m, destroy_count_t(down))); + pump(q); + m.reset(); + BOOST_TEST_EQ(q.size(), 1u); + capy::abandon(q.front()); + q.pop(); + pump(q); + BOOST_TEST_EQ(down, 1); + } + + // The lock handed to a waiter that is destroyed before it resumes + // passes on instead of staying held. + void testHandoffToDestroyedWaiterUnlocks() + { + async_mutex m; + BOOST_TEST(m.lock().await_ready()); + std::queue q; + auto waiter = [](async_mutex& mx) -> task + { + std::ignore = co_await mx.lock(); + mx.unlock(); + }; + run_async(queuing_executor(q))(waiter(m)); + pump(q); // waiter queued on the mutex + m.unlock(); // hands the lock over, posts the waiter + BOOST_TEST_EQ(q.size(), 1u); + capy::abandon(q.front()); // torn down before it resumes + q.pop(); + pump(q); + BOOST_TEST(!m.is_locked()); + } + + // A waiter claimed by its stop request belongs to the executor's + // queue: discarding it after the mutex is gone touches nothing. + void testStopClaimedWaiterOutlivesMutex() + { + auto p = std::make_unique(); + std::queue q; + std::stop_source ss; + // Held by the test, not a coroutine: nothing unlocks it later, + // and unlocking first would pop the claimed waiter before the + // destructor could see it. + BOOST_TEST(p->lock().await_ready()); + int down = 0; + auto waiter = [](async_mutex& x, destroy_count_t) -> task + { + std::ignore = co_await x.lock(); + }; + run_async(queuing_executor(q), ss.get_token())( + waiter(*p, destroy_count_t(down))); + pump(q); + ss.request_stop(); + BOOST_TEST_EQ(q.size(), 1u); + p.reset(); + capy::abandon(q.front()); + q.pop(); + pump(q); + BOOST_TEST_EQ(down, 1); + } + void run() { + testStopClaimedWaiterOutlivesMutex(); + testDestroyMutexCancelsWaiters(); + testCanceledWaiterDestroyedAtTeardown(); + testHandoffChainDoesNotRecurse(); + testUnlockWhenPostDestroysOwner(); + testOwnerTornDownWhileHoldingLock(); + testHandoffToDestroyedWaiterUnlocks(); testAcquireUnlocked(); testContendedLock(); testCancellation(); diff --git a/test/unit/ex/async_waker.cpp b/test/unit/ex/async_waker.cpp index 97e2b2638..44f05d914 100644 --- a/test/unit/ex/async_waker.cpp +++ b/test/unit/ex/async_waker.cpp @@ -11,19 +11,117 @@ #include #include +#include #include #include #include +#include #include +#include +#include +#include +#include +#include #include #include +#include -#include "test_suite.hpp" +#include "test_helpers.hpp" namespace boost { namespace capy { +namespace { + +// One thread that resumes what is posted to it, and reports when it +// has nothing left to run. +class worker +{ + std::mutex m_; + std::condition_variable cv_; + std::deque q_; + bool running_ = false; + bool stop_ = false; + std::thread t_; + + void loop() + { + std::unique_lock lk(m_); + for(;;) + { + cv_.wait(lk, [&]{ return stop_ || !q_.empty(); }); + if(q_.empty()) + return; + auto c = q_.front(); + q_.pop_front(); + running_ = true; + lk.unlock(); + c.h.resume(); + lk.lock(); + running_ = false; + } + } + +public: + ~worker() + { + { + std::lock_guard lk(m_); + stop_ = true; + } + cv_.notify_one(); + t_.join(); + } + + worker() + : t_([this]{ loop(); }) + { + } + + bool idle() + { + std::lock_guard lk(m_); + return q_.empty() && !running_; + } + + void push(continuation const& c) + { + { + std::lock_guard lk(m_); + q_.push_back(c); + } + cv_.notify_one(); + } +}; + +struct worker_executor +{ + worker* w_; + + bool operator==(worker_executor const& o) const noexcept + { + return w_ == o.w_; + } + test_io_context& context() const noexcept + { + return default_test_io_context(); + } + void on_work_started() const noexcept {} + void on_work_finished() const noexcept {} + std::coroutine_handle<> dispatch(continuation& c) const + { + w_->push(c); + return std::noop_coroutine(); + } + void post(continuation& c) const + { + w_->push(c); + } +}; + +} // namespace + struct async_waker_test { // wake() before wait(): the token is latched and the @@ -253,8 +351,262 @@ struct async_waker_test } } + // A waiting frame destroyed without resuming leaves the waker + // empty, so a later wake latches a token for the next wait. + void testDestroyWaitingFrame() + { + async_waker waker; + std::queue q; + queuing_executor ex(q); + int down = 0; + auto t = [](async_waker& wk, destroy_count_t) -> task + { + std::ignore = co_await wk.wait(); + }(waker, destroy_count_t(down)); + auto h = t.handle(); + t.release(); + io_env env; + env.executor = executor_ref(ex); + h.promise().set_environment(&env); + h.resume(); + BOOST_TEST_EQ(down, 0); + h.destroy(); + BOOST_TEST_EQ(down, 1); + + waker.wake(); + BOOST_TEST(q.empty()); + std::error_code ec = make_error_code(error::canceled); + auto next = [](async_waker& wk, std::error_code& out) -> task + { + auto [e] = co_await wk.wait(); + out = e; + }; + run_async(ex)(next(waker, ec)); + pump(q); + BOOST_TEST(!ec); + } + + // A destroyed waker can no longer wake its waiter, so the waiter + // resumes with an error instead of disappearing. + void testDestroyWakerCancelsWaiter() + { + int down = 0; + std::error_code ec; + auto w = std::make_unique(); + auto waiter = [](async_waker& wk, std::error_code& out, + destroy_count_t) -> task + { + auto [e] = co_await wk.wait(); + out = e; + }; + std::queue q; + run_async(queuing_executor(q))(waiter(*w, ec, destroy_count_t(down))); + pump(q); + w.reset(); + BOOST_TEST_EQ(down, 0); + BOOST_TEST_EQ(q.size(), 1u); + pump(q); + BOOST_TEST(ec == make_error_code(error::canceled)); + BOOST_TEST_EQ(down, 1); + } + + // A stop requested while the waiter is suspending is never lost: + // either the stop callback or the waiter's own re-check cancels it. + void testStopRacesWait() + { + thread_pool pool(1); + async_waker w; + std::atomic done{0}; + auto waiter = [](async_waker& x, std::atomic& d, + std::atomic& en) -> task + { + en.store(true); + std::ignore = co_await x.wait(); + d.fetch_add(1); + }; + for(int i = 0; i < 2000; ++i) + { + std::stop_source ss; + std::atomic entered{false}; + run_async(pool.get_executor(), ss.get_token(), + std::allocator{})(waiter(w, done, entered)); + while(!entered.load()) + std::this_thread::yield(); + ss.request_stop(); + // Only a watchdog: a lost cancel never completes. + auto const limit = std::chrono::steady_clock::now() + + std::chrono::seconds(30); + while(done.load() != i + 1 && + std::chrono::steady_clock::now() < limit) + std::this_thread::yield(); + if(!BOOST_TEST_EQ(done.load(), i + 1)) + break; + } + pool.join(); + } + + // A woken waiter torn down before it resumes, then the waker. + void testWokenWaiterDestroyedThenWakerDestroyed() + { + int down = 0; + auto p = std::make_unique(); + std::queue q; + auto waiter = [](async_waker& x, destroy_count_t) -> task + { + std::ignore = co_await x.wait(); + }; + run_async(queuing_executor(q), std::allocator{})( + waiter(*p, destroy_count_t(down))); + pump(q); + p->wake(); + BOOST_TEST_EQ(q.size(), 1u); + capy::abandon(q.front()); + q.pop(); + pump(q); + p.reset(); + BOOST_TEST_EQ(down, 1); + } + + // The waker may die on the waking thread right after wake(), while + // the waiter resumes on its own thread. + void testWakerDestroyedWhileWokenWaiterResumes() + { + worker wk; + worker_executor ex{&wk}; + for(int i = 0; i < 300; ++i) + { + auto w = std::make_unique(); + std::atomic entered{false}; + std::atomic done{false}; + auto waiter = [](async_waker& x, std::atomic& en, + std::atomic& d) -> task + { + en.store(true); + std::ignore = co_await x.wait(); + d.store(true); + }; + run_async(ex, std::allocator{})( + waiter(*w, entered, done)); + // Entered and then idle: suspended on the waker. + while(!entered.load() || !wk.idle()) + std::this_thread::yield(); + w->wake(); + w.reset(); + // Only a watchdog: a waiter that is never resumed hangs. + auto const limit = std::chrono::steady_clock::now() + + std::chrono::seconds(30); + while(!done.load() && + std::chrono::steady_clock::now() < limit) + std::this_thread::yield(); + if(!BOOST_TEST(done.load())) + break; + } + } + + // A stop requested after the waker that woke the waiter is gone + // finds no callback reaching into it. + void testStopAfterWakeAndWakerDestroyed() + { + std::error_code ec = make_error_code(error::canceled); + auto w = std::make_unique(); + std::stop_source ss; + auto waiter = [](async_waker& wk, std::error_code& out) -> task + { + auto [e] = co_await wk.wait(); + out = e; + }; + std::queue q; + run_async(queuing_executor(q), ss.get_token(), + std::allocator{})(waiter(*w, ec)); + pump(q); + w->wake(); + BOOST_TEST_EQ(q.size(), 1u); + w.reset(); + ss.request_stop(); + BOOST_TEST_EQ(q.size(), 1u); + pump(q); + BOOST_TEST(!ec); + } + + // The waiter's stop callback reaches into the waker, so a stop + // requested after the waker is gone must find it deregistered. + void testStopAfterWakerDestroyed() + { + std::error_code ec; + auto w = std::make_unique(); + std::stop_source ss; + auto waiter = [](async_waker& wk, std::error_code& out) -> task + { + auto [e] = co_await wk.wait(); + out = e; + }; + std::queue q; + run_async(queuing_executor(q), ss.get_token())(waiter(*w, ec)); + pump(q); + w.reset(); + ss.request_stop(); + BOOST_TEST_EQ(q.size(), 1u); + pump(q); + BOOST_TEST(ec == make_error_code(error::canceled)); + } + + // A waiter claimed by its stop request belongs to the executor's + // queue: discarding it after the waker is gone touches nothing. + void testStopClaimedWaiterOutlivesWaker() + { + auto p = std::make_unique(); + std::queue q; + std::stop_source ss; + int down = 0; + auto waiter = [](async_waker& x, destroy_count_t) -> task + { + std::ignore = co_await x.wait(); + }; + run_async(queuing_executor(q), ss.get_token())( + waiter(*p, destroy_count_t(down))); + pump(q); + ss.request_stop(); + BOOST_TEST_EQ(q.size(), 1u); + p.reset(); + capy::abandon(q.front()); + q.pop(); + pump(q); + BOOST_TEST_EQ(down, 1); + } + + // Same for a waiter claimed by wake(). + void testWakeClaimedWaiterOutlivesWaker() + { + auto p = std::make_unique(); + std::queue q; + int down = 0; + auto waiter = [](async_waker& x, destroy_count_t) -> task + { + std::ignore = co_await x.wait(); + }; + run_async(queuing_executor(q))( + waiter(*p, destroy_count_t(down))); + pump(q); + p->wake(); + BOOST_TEST_EQ(q.size(), 1u); + p.reset(); + capy::abandon(q.front()); + q.pop(); + pump(q); + BOOST_TEST_EQ(down, 1); + } + void run() { + testStopClaimedWaiterOutlivesWaker(); + testWakeClaimedWaiterOutlivesWaker(); + testDestroyWakerCancelsWaiter(); + testDestroyWaitingFrame(); + testStopAfterWakerDestroyed(); + testStopAfterWakeAndWakerDestroyed(); + testWokenWaiterDestroyedThenWakerDestroyed(); + testStopRacesWait(); + testWakerDestroyedWhileWokenWaiterResumes(); testLatchedTokenBeforeWait(); testWakeAfterWaitResumes(); testCrossThreadWake(); diff --git a/test/unit/ex/execution_context.cpp b/test/unit/ex/execution_context.cpp index 8eb128e11..040ff74b6 100644 --- a/test/unit/ex/execution_context.cpp +++ b/test/unit/ex/execution_context.cpp @@ -285,6 +285,90 @@ struct execution_context_test BOOST_TEST(shutdown_called); } + // A service created after its context shut down starts shut down, + // so it never holds work that its context can no longer run. + void + testServiceAfterShutdownStartsShutDown() + { + static int shutdowns; + shutdowns = 0; + struct counted_service : execution_context::service + { + explicit counted_service(execution_context&) {} + void shutdown() override { ++shutdowns; } + }; + bool made_shut_down = false; + { + closable_test_context ctx; + ctx.shutdown(); + ctx.make_service(made_shut_down); + BOOST_TEST(made_shut_down); + ctx.use_service(); + BOOST_TEST_EQ(shutdowns, 1); + } + // Not shut down a second time when the context is destroyed. + BOOST_TEST_EQ(shutdowns, 1); + } + + // A service's destructor still finds the older services, which are + // destroyed after it. + void + testLiveServiceVisibleDuringDestroy() + { + static execution_context::service* seen; + seen = nullptr; + struct registry_service : execution_context::service + { + explicit registry_service(execution_context&) {} + void shutdown() override {} + }; + struct client_service : execution_context::service + { + execution_context* ctx; + explicit client_service(execution_context& c) : ctx(&c) + { + ctx->use_service(); + } + ~client_service() + { + seen = ctx->find_service(); + } + void shutdown() override {} + }; + { + test_io_context ctx; + ctx.use_service(); + } + BOOST_TEST(seen != nullptr); + } + + // A service created while the context destroys its services is + // destroyed too, not leaked. + void + testServiceCreatedDuringDestroy() + { + static int late_down; + late_down = 0; + struct late_service : execution_context::service + { + explicit late_service(execution_context&) {} + ~late_service() { ++late_down; } + void shutdown() override {} + }; + struct older_service : execution_context::service + { + execution_context* ctx; + explicit older_service(execution_context& c) : ctx(&c) {} + ~older_service() { ctx->use_service(); } + void shutdown() override {} + }; + { + test_io_context ctx; + ctx.make_service(); + } + BOOST_TEST_EQ(late_down, 1); + } + void testMultipleServices() { @@ -438,6 +522,9 @@ struct execution_context_test void run() { + testServiceAfterShutdownStartsShutDown(); + testServiceCreatedDuringDestroy(); + testLiveServiceVisibleDuringDestroy(); testConstruct(); testHasService(); testFindService(); diff --git a/test/unit/ex/frame_allocator.cpp b/test/unit/ex/frame_allocator.cpp index 93c71123a..82d755827 100644 --- a/test/unit/ex/frame_allocator.cpp +++ b/test/unit/ex/frame_allocator.cpp @@ -10,6 +10,14 @@ // Test that header file is self-contained. #include +#include "test_suite.hpp" + +#include +#include +#include +#include +#include + /* Frame Allocator System ====================== @@ -106,3 +114,94 @@ Destruction order (LIFO): This guarantees frame_memory_resource outlives all frames it allocated. */ + +namespace boost { +namespace capy { + +namespace { + +struct hop_coro +{ + struct promise_type + { + hop_coro get_return_object() + { + return hop_coro{ + std::coroutine_handle::from_promise(*this)}; + } + + std::suspend_always initial_suspend() noexcept { return {}; } + std::suspend_always final_suspend() noexcept { return {}; } + void return_void() {} + void unhandled_exception() { std::terminate(); } + }; + + std::coroutine_handle h_; + + ~hop_coro() + { + if(h_) + h_.destroy(); + } + + hop_coro(hop_coro&& o) noexcept + : h_(std::exchange(o.h_, nullptr)) + { + } + + hop_coro& operator=(hop_coro&&) = delete; + hop_coro(hop_coro const&) = delete; + +private: + explicit hop_coro(std::coroutine_handle h) + : h_(h) + { + } +}; + +// Sets the frame allocator, suspends, then sets it again after +// being resumed, possibly on another thread. +hop_coro +make_hop( + std::pmr::memory_resource* before, + std::pmr::memory_resource* after) +{ + set_current_frame_allocator(before); + co_await std::suspend_always{}; + set_current_frame_allocator(after); +} + +} // anonymous namespace + +struct frame_allocator_test +{ + void + testSetAfterResumeOnAnotherThread() + { + auto* before = std::pmr::null_memory_resource(); + auto* after = std::pmr::new_delete_resource(); + + auto coro = make_hop(before, after); + + // The thread exits before the second resumption, so a write + // through its slot would also land in freed storage. + std::thread([&] { coro.h_.resume(); }).join(); + + set_current_frame_allocator(nullptr); + coro.h_.resume(); + BOOST_TEST(get_current_frame_allocator() == after); + + set_current_frame_allocator(nullptr); + } + + void + run() + { + testSetAfterResumeOnAnotherThread(); + } +}; + +TEST_SUITE(frame_allocator_test, "capy.frame_allocator"); + +} // capy +} // boost diff --git a/test/unit/ex/recycling_memory_resource.cpp b/test/unit/ex/recycling_memory_resource.cpp index c9a450234..d1bbdb2ba 100644 --- a/test/unit/ex/recycling_memory_resource.cpp +++ b/test/unit/ex/recycling_memory_resource.cpp @@ -108,12 +108,47 @@ class recycling_memory_resource_test BOOST_TEST_PASS(); } + // A thread_local made before the thread's pool was first used is + // destroyed after that pool's exit-time drain. Blocks it allocates + // and frees then are not left in the drained pool, where nothing + // releases them. + void + testUseAfterThreadExitDrain() + { + constexpr std::size_t bytes = 128; // size class 1 + constexpr std::size_t align = 8; + + std::thread([] { + struct late_user + { + ~late_user() + { + recycling_memory_resource mr; + void* p = mr.allocate_fast(bytes, align); + mr.deallocate_fast(p, bytes, align); + } + }; + thread_local late_user user; + + // Fill both the thread's pool and the shared one. + recycling_memory_resource mr; + std::vector ptrs; + for(int i = 0; i < 40; ++i) + ptrs.push_back(mr.allocate_fast(bytes, align)); + for(void* p : ptrs) + mr.deallocate_fast(p, bytes, align); + }).join(); + + BOOST_TEST_PASS(); + } + void run() { testIsEqual(); testSlowPaths(); testThreadExitTeardown(); + testUseAfterThreadExitDrain(); } }; diff --git a/test/unit/ex/run.cpp b/test/unit/ex/run.cpp index 5c67645ef..cbd3fe5c4 100644 --- a/test/unit/ex/run.cpp +++ b/test/unit/ex/run.cpp @@ -11,12 +11,14 @@ #include #include +#include #include #include #include #include #include "test/unit/custom_task.hpp" +#include "test/unit/death_test.hpp" #include "test/unit/test_helpers.hpp" #include @@ -25,7 +27,11 @@ #include #include +#include +#include +#include #include +#include namespace boost { namespace capy { @@ -37,6 +43,39 @@ static_assert(IoAwaitable, true>>); using test::custom_task; +// Destroys its frame even while lent, as a non-conforming IoRunnable +// may. +struct lax_task : custom_task +{ + explicit lax_task(custom_task&& t) noexcept + : custom_task(std::move(t)) + { + } + + lax_task(lax_task&&) noexcept = default; + + ~lax_task() + { + if(auto h = handle()) + { + release(); + h.destroy(); + } + } +}; + +static_assert(IoRunnable); + +// custom_task passes no environment to what it awaits. +struct park_plain +{ + parked* slot; + + bool await_ready() const noexcept { return false; } + void await_suspend(std::coroutine_handle<> h) noexcept { slot->h = h; } + void await_resume() const noexcept {} +}; + // Test allocator that wraps std::allocator template struct test_allocator @@ -63,6 +102,12 @@ struct test_allocator // run Tests //---------------------------------------------------------- +task +make_throwing_int_task() +{ + throw_test_exception("argument"); +} + struct run_test { static custom_task @@ -251,6 +296,63 @@ struct run_test BOOST_TEST_EQ(result, 123); } + // A task argument that throws leaves the caller's frame allocator + // installed, not the destroyed wrapper's. + void + testTaskArgumentThrowsKeepsAllocator() + { + int dispatch_count = 0; + test_executor ex(1, dispatch_count); + test_allocator alloc; + std::pmr::monotonic_buffer_resource mr; + int checked = 0; + + auto outer = [&]() -> task<> { + auto* const before = get_current_frame_allocator(); + auto check = [&] { + BOOST_TEST(get_current_frame_allocator() == before); + ++checked; + }; + try + { + co_await capy::run(ex, alloc)(make_throwing_int_task()); + } + catch(test_exception const&) + { + } + check(); + BOOST_TEST_EQ(co_await returns_int(), 123); + try + { + co_await capy::run(alloc)(make_throwing_int_task()); + } + catch(test_exception const&) + { + } + check(); + BOOST_TEST_EQ(co_await returns_int(), 123); + try + { + co_await capy::run(ex, &mr)(make_throwing_int_task()); + } + catch(test_exception const&) + { + } + check(); + try + { + co_await capy::run(&mr)(make_throwing_int_task()); + } + catch(test_exception const&) + { + } + check(); + }; + + run_async(ex)(outer()); + BOOST_TEST_EQ(checked, 4); + } + void testExWithStopTokenAndAllocator() { @@ -531,6 +633,502 @@ struct run_test compute_pool.join(); } + // Executor that queues posts and counts outstanding work. + struct counting_queue_executor + { + std::queue* q_; + int* work_; + + bool operator==(counting_queue_executor const& o) const noexcept + { + return q_ == o.q_; + } + test_io_context& context() const noexcept + { + return default_test_io_context(); + } + void on_work_started() const noexcept { ++*work_; } + void on_work_finished() const noexcept { --*work_; } + std::coroutine_handle<> dispatch(continuation& c) const + { + q_->push(c); + return std::noop_coroutine(); + } + void post(continuation& c) const + { + q_->push(c); + } + }; + + // An inner chain torn down from below under run(mr) is freed + // before the frame owning the resource is destroyed. + void testInnerTeardownFreesBeforeResourceOwner() + { + lease_resource mr; + int child_down = 0, owner_down = 0, n = 0; + parked slot; + test_executor caller(n); + auto child = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto owner = [&](destroy_count_t) -> task + { + lease l(mr); + co_await capy::run(&mr)(child(&slot, destroy_count_t(child_down))); + }; + run_async(caller)(owner(destroy_count_t(owner_down))); + BOOST_TEST(slot); + capy::abandon(slot); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(owner_down, 1); + BOOST_TEST(mr.allocs.load() > 0); + BOOST_TEST(mr.balanced()); + } + + // Tearing down a chain parked under run(mr) leaves the tearing + // thread's frame allocator as it was. + void testTeardownKeepsThreadAllocator() + { + lease_resource mr; + std::pmr::monotonic_buffer_resource mine; + int n = 0; + parked slot; + test_executor caller(n); + auto child = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto owner = [&]() -> task + { + co_await capy::run(&mr)(child(&slot)); + }; + run_async(caller)(owner()); + BOOST_TEST(slot); + auto* const before = get_current_frame_allocator(); + set_current_frame_allocator(&mine); + capy::abandon(slot); + BOOST_TEST(get_current_frame_allocator() == &mine); + set_current_frame_allocator(before); + } + + // The same across an executor hop. + void testInnerTeardownFreesBeforeResourceOwnerEx() + { + lease_resource mr; + std::queue q; + int child_down = 0, owner_down = 0, n = 0; + parked slot; + queuing_executor caller(q); + test_executor inner(n); + auto child = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto owner = [&](destroy_count_t) -> task + { + lease l(mr); + co_await capy::run(inner, &mr)( + child(&slot, destroy_count_t(child_down))); + }; + run_async(caller)(owner(destroy_count_t(owner_down))); + pump(q); + BOOST_TEST(slot); + capy::abandon(slot); + pump(q); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(owner_down, 1); + BOOST_TEST(mr.allocs.load() > 0); + BOOST_TEST(mr.balanced()); + } + + // A run that gives the inner chain its own allocator frees all of + // that chain when it is torn down from below. + void testInnerTeardownWithRunAllocator() + { + alloc_counts st; + int child_down = 0, parent_down = 0, n = 0; + parked slot; + { + test_executor caller(n); + auto child = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await capy::run(counting_allocator(st))( + child(&slot, destroy_count_t(child_down))); + }; + run_async(caller)(parent(destroy_count_t(parent_down))); + BOOST_TEST(slot); + capy::abandon(slot); + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // The same across an executor hop: the inner chain is freed, then + // the parent goes on the caller's executor. + void testInnerTeardownWithRunExAllocator() + { + alloc_counts st; + std::queue q; + int child_down = 0, parent_down = 0, n = 0; + parked slot; + { + queuing_executor caller(q); + test_executor inner(n); + auto child = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await capy::run(inner, counting_allocator(st))( + child(&slot, destroy_count_t(child_down))); + }; + run_async(caller)(parent(destroy_count_t(parent_down))); + pump(q); + BOOST_TEST(slot); + capy::abandon(slot); + pump(q); + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // Tearing down the inner chain posts the parent's destruction to + // the caller's executor; running it destroys the parent there. + void testInnerTeardownReapsParentOnCallerExecutor() + { + std::queue q; + int work = 0, child_down = 0, parent_down = 0, n = 0; + parked slot; + counting_queue_executor caller{&q, &work}; + test_executor inner(n); + auto child = [](parked* s, destroy_count_t) -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await capy::run(inner)(child(&slot, destroy_count_t(child_down))); + }; + run_async(caller)(parent(destroy_count_t(parent_down))); + pump(q); + BOOST_TEST(slot); + BOOST_TEST_EQ(work, 2); // run_async's guard and run(ex)'s hold + + capy::abandon(slot); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 0); // not inline: queued on caller + BOOST_TEST_EQ(q.size(), 1u); + + pump(q); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST_EQ(work, 0); + } + + // A caller whose context is shut down unwinds the parent chain as + // the unwind is posted, destroying the allocator its root frame + // holds; the boundary frame must not be freed into it after. + void testCallerUnwoundInlineWithValueAllocator() + { + struct closing_executor + { + std::queue* q_; + bool* closed_; + + bool operator==(closing_executor const& o) const noexcept + { + return q_ == o.q_; + } + test_io_context& context() const noexcept + { + return default_test_io_context(); + } + void on_work_started() const noexcept {} + void on_work_finished() const noexcept {} + std::coroutine_handle<> dispatch(continuation& c) const + { + post(c); + return std::noop_coroutine(); + } + void post(continuation& c) const + { + if(*closed_) + capy::abandon(c); + else + q_->push(c); + } + }; + alloc_counts st; + std::queue q; + bool closed = false; + int child_down = 0, parent_down = 0, n = 0; + parked slot; + closing_executor caller{&q, &closed}; + test_executor inner(n); + auto child = [](parked* s, destroy_count_t) -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await capy::run(inner)(child(&slot, destroy_count_t(child_down))); + }; + run_async(caller, counting_allocator(st))( + parent(destroy_count_t(parent_down))); + pump(q); + BOOST_TEST(slot); + closed = true; + capy::abandon(slot); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // An inner executor that can no longer run the unwind destroys the + // boundary instead; the inner task and the parent still come down. + void testClosedInnerExecutorDestroysBoundary() + { + std::queue q, iq; + bool closed = false; + int work = 0, child_down = 0, parent_down = 0; + parked slot; + counting_queue_executor caller{&q, &work}; + closing_executor inner{&iq, &closed}; + auto child = [](parked* s, destroy_count_t) -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await capy::run(inner)( + child(&slot, destroy_count_t(child_down))); + }; + run_async(caller)(parent(destroy_count_t(parent_down))); + while(!q.empty() || !iq.empty()) + { + pump(q); + pump(iq); + } + BOOST_TEST(slot); + + closed = true; + capy::abandon(slot); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST(iq.empty()); + BOOST_TEST_EQ(parent_down, 0); + BOOST_TEST_EQ(q.size(), 1u); + + pump(q); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST_EQ(work, 0); + } + + // A caller context that drains the posted unwind instead of running + // it still destroys the parent. + void testDrainedUnwindDestroysParent() + { + std::queue q; + int work = 0, child_down = 0, parent_down = 0, n = 0; + parked slot; + counting_queue_executor caller{&q, &work}; + test_executor inner(n); + auto child = [](parked* s, destroy_count_t) -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await capy::run(inner)( + child(&slot, destroy_count_t(child_down))); + }; + run_async(caller)(parent(destroy_count_t(parent_down))); + pump(q); + capy::abandon(slot); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(q.size(), 1u); + capy::abandon(q.front()); + q.pop(); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST_EQ(work, 0); + BOOST_TEST(q.empty()); + } + + // run(ex) holds work on the caller's executor until the parent is + // resumed. + void testRunHoldsCallerWork() + { + std::queue q; + int work = 0, n = 0; + parked slot; + counting_queue_executor caller{&q, &work}; + test_executor inner(n); + auto child = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&]() -> task + { + co_await capy::run(inner)(child(&slot)); + }; + run_async(caller)(parent()); + pump(q); + BOOST_TEST_EQ(work, 2); + slot.resume(); // child finishes, parent posted + BOOST_TEST_EQ(work, 2); // still held while the parent is queued + BOOST_TEST_EQ(q.size(), 1u); + q.front().h.resume(); // parent resumes, run(ex) releases + q.pop(); + BOOST_TEST_EQ(work, 0); + pump(q); + BOOST_TEST_EQ(work, 0); + } + + // A caller context that drains the parent instead of resuming it + // still releases run(ex)'s hold. + void testDrainedParentReleasesWork() + { + std::queue q; + int work = 0, parent_down = 0, n = 0; + parked slot; + counting_queue_executor caller{&q, &work}; + test_executor inner(n); + auto child = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await capy::run(inner)(child(&slot)); + }; + run_async(caller)(parent(destroy_count_t(parent_down))); + pump(q); + slot.resume(); + BOOST_TEST_EQ(q.size(), 1u); + capy::abandon(q.front()); + q.pop(); + pump(q); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST_EQ(work, 0); + } + + struct throwing_post_executor + { + bool operator==(throwing_post_executor const&) const noexcept + { + return true; + } + test_io_context& context() const noexcept + { + return default_test_io_context(); + } + void on_work_started() const noexcept {} + void on_work_finished() const noexcept {} + std::coroutine_handle<> dispatch(continuation&) const + { + throw std::runtime_error("dispatch"); + } + void post(continuation&) const + { + throw std::runtime_error("post"); + } + }; + + // An executor refusing the task makes await_suspend throw with + // nothing lent and no work held on the caller's executor. Called + // by hand: a task wraps each co_await in a noexcept await_suspend, + // so inside a task the throw terminates instead. + void testRunExPostThrows() + { + std::queue q; + int work = 0, child_down = 0; + bool caught = false; + counting_queue_executor caller{&q, &work}; + io_env env{executor_ref(caller), {}, nullptr}; + auto child = [](destroy_count_t) -> task { co_return; }; + { + auto aw = capy::run(throwing_post_executor{})( + child(destroy_count_t(child_down))); + try + { + std::ignore = aw.await_suspend(std::noop_coroutine(), &env); + } + catch(std::runtime_error const&) + { + caught = true; + } + BOOST_TEST_EQ(work, 0); + BOOST_TEST_EQ(child_down, 0); + } + BOOST_TEST(caught); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST(q.empty()); + } + +#if !defined(_WIN32) + // Start a parent by hand without lending it, so destroying its + // task object destroys it from above. + template + static void + destroy_from_above(F parent) + { + int n = 0; + test_executor caller(n); + io_env env{executor_ref(caller), {}, nullptr}; + auto t = parent(); + t.handle().promise().set_environment(&env); + t.handle().resume(); + } + + // Destroying a parent from above while run(ex) has its task lent + // terminates, whatever the task type does when destroyed. + void testDestroyParentAwaitingRunExTerminates() + { + BOOST_TEST(terminates([] { + parked slot; + int n = 0; + test_executor inner(n); + auto child = [](parked* s) -> custom_task + { + co_await park_plain{s}; + }; + destroy_from_above([&]() -> task + { + co_await capy::run(inner)(lax_task(child(&slot))); + }); + })); + } + + void testDestroyParentAwaitingRunTerminates() + { + BOOST_TEST(terminates([] { + parked slot; + auto child = [](parked* s) -> custom_task + { + co_await park_plain{s}; + }; + destroy_from_above([&]() -> task + { + co_await capy::run(std::stop_token{})( + lax_task(child(&slot))); + }); + })); + } +#endif + void run() { @@ -555,6 +1153,23 @@ struct run_test testAllocatorPropagationThroughRun(); testRunExStrandFirstInstruction(); testHopsBackToIoThread(); + testTaskArgumentThrowsKeepsAllocator(); + testTeardownKeepsThreadAllocator(); + testCallerUnwoundInlineWithValueAllocator(); + testInnerTeardownReapsParentOnCallerExecutor(); + testInnerTeardownWithRunAllocator(); + testInnerTeardownFreesBeforeResourceOwner(); + testInnerTeardownFreesBeforeResourceOwnerEx(); + testInnerTeardownWithRunExAllocator(); + testDrainedUnwindDestroysParent(); + testClosedInnerExecutorDestroysBoundary(); + testRunHoldsCallerWork(); + testDrainedParentReleasesWork(); + testRunExPostThrows(); +#if !defined(_WIN32) + testDestroyParentAwaitingRunExTerminates(); + testDestroyParentAwaitingRunTerminates(); +#endif } }; diff --git a/test/unit/ex/run_async.cpp b/test/unit/ex/run_async.cpp index 3536f7f86..44f94c6da 100644 --- a/test/unit/ex/run_async.cpp +++ b/test/unit/ex/run_async.cpp @@ -11,6 +11,7 @@ #include #include +#include #include #include @@ -107,11 +108,11 @@ static_assert(Executor); /// Queuing executor - queues for manual execution. struct queue_executor { - std::queue>* queue_; + std::queue* queue_; test_io_context* ctx_ = nullptr; static test_io_context default_ctx_; - explicit queue_executor(std::queue>& q) + explicit queue_executor(std::queue& q) : queue_(&q) { } @@ -131,13 +132,13 @@ struct queue_executor std::coroutine_handle<> dispatch(continuation& c) const { - queue_->push(c.h); + queue_->push(c); return std::noop_coroutine(); } void post(continuation& c) const { - queue_->push(c.h); + queue_->push(c); } }; @@ -159,8 +160,99 @@ using test::custom_task; // run_async Tests //---------------------------------------------------------- +// Counts outstanding work; dispatch and post always throw. +struct refusing_executor +{ + int* work_; + + bool operator==(refusing_executor const& other) const noexcept + { + return work_ == other.work_; + } + + test_io_context& context() const noexcept + { + return default_test_io_context(); + } + + void on_work_started() const noexcept { ++*work_; } + void on_work_finished() const noexcept { --*work_; } + + std::coroutine_handle<> dispatch(continuation&) const + { + throw_test_exception("dispatch"); + } + + void post(continuation&) const + { + throw_test_exception("post"); + } +}; + +task<> +make_throwing_task() +{ + throw_test_exception("argument"); +} + struct run_async_test { + // A task drained before it starts tears down like a parked one: + // the completion handlers are destroyed without being called, and + // its frame is freed. + void testTaskDrainedBeforeStart() + { + alloc_counts st; + int task_down = 0, handler_down = 0; + { + std::queue q; + queuing_executor ex(q); + auto probe = [](destroy_count_t) -> task { co_return; }; + run_async(ex, counting_allocator(st), + [d = destroy_count_t(handler_down)] { BOOST_TEST(false); }, + [](std::exception_ptr) { BOOST_TEST(false); })( + probe(destroy_count_t(task_down))); + BOOST_TEST_EQ(q.size(), 1u); + auto h = q.front(); + q.pop(); + capy::abandon(h); + pump(q); + } + BOOST_TEST_EQ(task_down, 1); + BOOST_TEST_EQ(handler_down, 1); + BOOST_TEST(st.balanced()); + } + + // The same through a memory_resource the caller owns: the task's + // frame is freed before the handlers, which go with the launch. + void testTaskDrainedBeforeStartWithResource() + { + int task_down = 0, handler_down = 0; + lease_resource mr; + { + std::queue q; + queuing_executor ex(q); + auto probe = [](destroy_count_t) -> task { co_return; }; + run_async(ex, &mr, + [d = destroy_count_t(handler_down), l = lease(mr)] + { + BOOST_TEST(false); + }, + [](std::exception_ptr) { BOOST_TEST(false); })( + probe(destroy_count_t(task_down))); + BOOST_TEST_EQ(q.size(), 1u); + auto h = q.front(); + q.pop(); + capy::abandon(h); + pump(q); + } + BOOST_TEST_EQ(task_down, 1); + BOOST_TEST_EQ(handler_down, 1); + BOOST_TEST(mr.allocs.load() > 1); + BOOST_TEST_EQ(mr.allocs.load(), mr.deallocs.load()); + BOOST_TEST_EQ(mr.late.load(), 1); // the launch's own frame + } + //---------------------------------------------------------- // Basic Functionality //---------------------------------------------------------- @@ -450,7 +542,7 @@ struct run_async_test // Three tasks on the same executor: one with a scoped stop token, // two with the default (empty) token. Cancelling the scoped token // should only affect that task, not the others. - std::queue> queue; + std::queue queue; queue_executor d(queue); bool default_1_stopped = true; @@ -474,7 +566,7 @@ struct run_async_test while(!queue.empty()) { - auto h = queue.front(); + auto h = queue.front().h; queue.pop(); h.resume(); } @@ -593,7 +685,7 @@ struct run_async_test void testAsyncDispatcherBasic() { - std::queue> queue; + std::queue queue; queue_executor d(queue); int result = 0; @@ -606,7 +698,7 @@ struct run_async_test // Execute the queued work while(!queue.empty()) { - auto h = queue.front(); + auto h = queue.front().h; queue.pop(); h.resume(); } @@ -617,7 +709,7 @@ struct run_async_test void testAsyncDispatcherMultiple() { - std::queue> queue; + std::queue queue; queue_executor d(queue); int sum = 0; @@ -631,7 +723,7 @@ struct run_async_test // Execute all while(!queue.empty()) { - auto h = queue.front(); + auto h = queue.front().h; queue.pop(); h.resume(); } @@ -777,11 +869,40 @@ struct run_async_test //---------------------------------------------------------- + // A dispatch that throws gives the task back destroyed, leaving no + // work outstanding. + void + testDispatchThrows() + { + int work = 0; + int down = 0; + auto t = [](destroy_count_t) -> task<> { co_return; }( + destroy_count_t(down)); + BOOST_TEST_THROWS( + run_async(refusing_executor{&work})(std::move(t)), + test_exception); + BOOST_TEST_EQ(work, 0); + BOOST_TEST_EQ(down, 1); + } + + // A task argument that throws leaves no work outstanding. + void + testTaskArgumentThrows() + { + int work = 0; + BOOST_TEST_THROWS( + run_async(refusing_executor{&work})(make_throwing_task()), + test_exception); + BOOST_TEST_EQ(work, 0); + } + void run() { // Basic Functionality testNoHandlers(); + testDispatchThrows(); + testTaskArgumentThrows(); testValueAllocator(); testValueAllocatorException(); testResultHandler(); @@ -826,6 +947,8 @@ struct run_async_test // Custom Task Type testCustomTaskType(); testCustomTaskTypeVoid(); + testTaskDrainedBeforeStart(); + testTaskDrainedBeforeStartWithResource(); } }; diff --git a/test/unit/ex/run_blocking.cpp b/test/unit/ex/run_blocking.cpp index e6bd60c17..b72975ba7 100644 --- a/test/unit/ex/run_blocking.cpp +++ b/test/unit/ex/run_blocking.cpp @@ -10,13 +10,21 @@ // Test that header file is self-contained. #include +#include #include +#include #include +#include "test_helpers.hpp" #include "test_suite.hpp" +#include #include #include +#include +#include +#include +#include namespace boost { namespace capy { @@ -37,6 +45,152 @@ struct test_exception : std::runtime_error } }; +namespace { + +// Posts a continuation from its destructor, as a frame-local primitive +// does when it is torn down. +struct posts_on_destroy +{ + blocking_executor ex; + continuation* c; + + posts_on_destroy(blocking_executor e, continuation* cc) noexcept + : ex(e) + , c(cc) + { + } + + posts_on_destroy(posts_on_destroy&& o) noexcept + : ex(o.ex) + , c(std::exchange(o.c, nullptr)) + { + } + + ~posts_on_destroy() + { + if(c) + ex.post(*c); + } +}; + +task +posting_probe(destroy_count_t, posts_on_destroy) +{ + co_return; +} + +task +plain_probe(destroy_count_t) +{ + co_return; +} + +// Posts from its destructor, after the context's services shut down. +struct dtor_poster : execution_context::service +{ + blocking_executor ex_; + continuation* c_; + + dtor_poster(execution_context&, blocking_executor ex, continuation& c) + : ex_(ex) + , c_(&c) + { + } + + ~dtor_poster() + { + ex_.post(*c_); + } + + void shutdown() override {} +}; + +// Starts posting from another thread when the context's services shut +// down, while the context drains, and joins that thread when destroyed. +struct shutdown_poster : execution_context::service +{ + blocking_executor ex_; + std::vector* cs_; + std::thread t_; + + shutdown_poster( + execution_context&, + blocking_executor ex, + std::vector& cs) + : ex_(ex) + , cs_(&cs) + { + } + + ~shutdown_poster() + { + t_.join(); + } + + void shutdown() override + { + t_ = std::thread([this] + { + for(auto& c : *cs_) + ex_.post(c); + }); + } +}; + +// Dispatches a coroutine while its context shuts down. +struct shutdown_dispatcher : execution_context::service +{ + blocking_executor ex_; + continuation* c_; + + shutdown_dispatcher( + execution_context&, + blocking_executor ex, + continuation& c) + : ex_(ex) + , c_(&c) + { + } + + void shutdown() override + { + safe_resume(ex_.dispatch(*c_)); + } +}; + +// A second dtor_poster, so one can be created while the first exists. +struct late_dtor_poster : dtor_poster +{ + using dtor_poster::dtor_poster; +}; + +// Creates a late_dtor_poster on the context when destroyed. +struct poster_maker +{ + blocking_context* ctx; + continuation* c; + + poster_maker(blocking_context& x, continuation& cont) noexcept + : ctx(&x) + , c(&cont) + { + } + + poster_maker(poster_maker&& o) noexcept + : ctx(std::exchange(o.ctx, nullptr)) + , c(o.c) + { + } + + ~poster_maker() + { + if(ctx) + ctx->make_service(ctx->get_executor(), *c); + } +}; + +} // namespace + //---------------------------------------------------------- // run_blocking Tests //---------------------------------------------------------- @@ -300,6 +454,159 @@ struct run_blocking_test BOOST_TEST_EQ(result, 43); } + void + testDestroyedContextDestroysQueued() + { + int down = 0; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto t = probe(destroy_count_t(down)); + auto h = t.handle(); + t.release(); + { + blocking_context ctx; + continuation c{h}; + ctx.get_executor().post(c); + } + BOOST_TEST_EQ(down, 1); + } + + // Work posted from another thread while the context drains is + // neither lost nor torn from the queue. + void + testPostDuringDrain() + { + constexpr int n = 2000; + int down = 0; + auto probe = [](destroy_count_t) -> task { co_return; }; + std::vector cs; + cs.reserve(n); + for(int i = 0; i < n; ++i) + { + auto t = probe(destroy_count_t(down)); + cs.push_back(continuation{t.handle()}); + t.release(); + } + { + blocking_context ctx; + ctx.make_service(ctx.get_executor(), cs); + } + BOOST_TEST_EQ(down, n); + } + + // Dispatch runs inline only from inside the context's own run(), so + // a context being shut down runs nothing dispatched to it. + void + testDispatchDuringShutdown() + { + int down = 0; + bool ran = false; + auto probe = [](destroy_count_t, bool& r) -> task + { + r = true; + co_return; + }; + auto t = probe(destroy_count_t(down), ran); + continuation c{t.handle()}; + t.release(); + { + blocking_context ctx; + ctx.make_service(ctx.get_executor(), c); + } + BOOST_TEST(!ran); + BOOST_TEST_EQ(down, 1); + } + + // An abandoned chain delivers no result: the loop ends with + // error::canceled instead. + void + testAbandonedChainThrows() + { + int down = 0; + bool called = false; + auto probe = [](destroy_count_t) -> task + { + co_await self_destroy_awaitable{}; + }; + std::error_code ec; + try + { + run_blocking([&]{ called = true; })(probe(destroy_count_t(down))); + } + catch(std::system_error const& e) + { + ec = e.code(); + } + BOOST_TEST(ec == error::canceled); + BOOST_TEST(!called); + BOOST_TEST_EQ(down, 1); + } + + // Work posted while the services are destroyed is destroyed too. + void + testPostFromServiceDestructor() + { + int down = 0; + auto t = plain_probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + { + blocking_context ctx; + ctx.make_service(ctx.get_executor(), c); + } + BOOST_TEST_EQ(down, 1); + } + + // A frame destroyed by the context's last drain may create a service + // whose destructor posts again; that work is destroyed too. + void + testServiceCreatedByLastDrainDestroyed() + { + int down = 0; + auto maker = [](poster_maker) -> task { co_return; }; + auto t = plain_probe(destroy_count_t(down)); + continuation probe_c{t.handle()}; + t.release(); + continuation maker_c; + { + blocking_context ctx; + auto m = maker(poster_maker(ctx, probe_c)); + maker_c.h = m.handle(); + m.release(); + ctx.make_service(ctx.get_executor(), maker_c); + } + BOOST_TEST_EQ(down, 1); + } + + // A frame its services destroy at shutdown can post to the context; + // that work is destroyed, not written into a freed queue. + void + testPostDuringServiceShutdown() + { + int down = 0; + int posted_down = 0; + // Outlives the context, so the queued work is given up by the + // context's shutdown rather than by the strand's destructor. + std::optional> s; + continuation c1; + continuation c2; + { + blocking_context ctx; + auto ex = ctx.get_executor(); + s.emplace(ex); + auto t2 = plain_probe(destroy_count_t(posted_down)); + c2.h = t2.handle(); + t2.release(); + auto t1 = posting_probe( + destroy_count_t(down), posts_on_destroy(ex, &c2)); + c1.h = t1.handle(); + t1.release(); + s->post(c1); + } + s.reset(); + BOOST_TEST_EQ(down, 1); + BOOST_TEST_EQ(posted_down, 1); + } + //---------------------------------------------------------- void @@ -331,6 +638,15 @@ struct run_blocking_test // Nested Tasks testNestedTask(); + + // Teardown + testDestroyedContextDestroysQueued(); + testDispatchDuringShutdown(); + testAbandonedChainThrows(); + testPostDuringDrain(); + testPostDuringServiceShutdown(); + testPostFromServiceDestructor(); + testServiceCreatedByLastDrainDestroyed(); } }; diff --git a/test/unit/ex/run_priority.cpp b/test/unit/ex/run_priority.cpp index b890936fc..23ece390e 100644 --- a/test/unit/ex/run_priority.cpp +++ b/test/unit/ex/run_priority.cpp @@ -79,17 +79,6 @@ make_log_coro(std::vector& log, std::string msg) }(&log, std::move(msg)); } -inline void -pump(std::queue>& q) -{ - while(!q.empty()) - { - auto h = q.front(); - q.pop(); - h.resume(); - } -} - } // namespace struct run_priority_test @@ -99,7 +88,7 @@ struct run_priority_test void testForwardCrossing() { - std::queue> q; + std::queue q; queuing_executor qe(q); test::priority_executor_state state; test::priority_executor pe(state, qe); @@ -146,7 +135,7 @@ struct run_priority_test void testReturnTripParentWrongFrame() { - std::queue> q; + std::queue q; queuing_executor qe(q); test::priority_executor_state state; test::priority_executor pe(state, qe); @@ -195,7 +184,7 @@ struct run_priority_test void testExitStrandOverPriority() { - std::queue> q; + std::queue q; queuing_executor qe(q); test::priority_executor_state state; test::priority_executor pe(state, qe); diff --git a/test/unit/ex/strand.cpp b/test/unit/ex/strand.cpp index 0415163b6..e81381195 100644 --- a/test/unit/ex/strand.cpp +++ b/test/unit/ex/strand.cpp @@ -16,16 +16,25 @@ #include #include +#include #include #include #include #include #include +#include "test_helpers.hpp" #include "test_suite.hpp" +#include #include #include +#include +#include +#include +#include +#include +#include #include #include @@ -40,19 +49,38 @@ static_assert(Executor>, static_assert(Executor>, "strand must satisfy Executor concept"); -// Helper to wait for a condition with timeout -template -bool wait_for(Pred pred, std::chrono::milliseconds timeout = std::chrono::milliseconds(5000)) +// Fails its next posts, running a hook in each failing post first. +struct reentrant_executor { - auto start = std::chrono::steady_clock::now(); - while(!pred()) + std::queue* q_; + test_io_context* ctx_; + int* fails_; + std::function* on_fail_; + + bool operator==(reentrant_executor const& o) const noexcept { - if(std::chrono::steady_clock::now() - start > timeout) - return false; - std::this_thread::sleep_for(std::chrono::milliseconds(1)); + return q_ == o.q_; } - return true; -} + test_io_context& context() const noexcept { return *ctx_; } + void on_work_started() const noexcept {} + void on_work_finished() const noexcept {} + std::coroutine_handle<> dispatch(continuation& c) const + { + post(c); + return std::noop_coroutine(); + } + void post(continuation& c) const + { + if(*fails_ > 0) + { + --*fails_; + if(*on_fail_) + std::exchange(*on_fail_, nullptr)(); + throw std::runtime_error("post"); + } + q_->push(c); + } +}; // Simple test coroutine that increments a counter struct counter_coro @@ -955,6 +983,90 @@ struct strand_test } } + // A started chain queued on a strand when its context shuts down + // comes down with every frame freed. + void + testShutdownDestroysQueuedParentedChain() + { + alloc_counts st; + int child_down = 0, parent_down = 0; + parked slot; + continuation c; + std::queue q; + { + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + strand s{qe}; + auto child = [](parked* sl, destroy_count_t) + -> task + { + co_await park_awaitable{sl}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await child(&slot, destroy_count_t(child_down)); + }; + run_async(s, counting_allocator(st))( + parent(destroy_count_t(parent_down))); + pump(q); + BOOST_TEST(slot); + + c = slot; + s.post(c); + // The strand's runner is lost; the chain stays queued. + while(!q.empty()) + { + capy::abandon(q.front()); + q.pop(); + } + BOOST_TEST_EQ(parent_down, 0); + ctx.reset(); + BOOST_TEST(q.empty()); + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.balanced()); + } + + // Service shutdown destroys what is still queued on every strand + // of the context, not only the first. + void + testShutdownDestroysQueuedAcrossStrands() + { + int down[4] = {}; + continuation conts[4] = {}; + std::queue q; + auto probe = [](destroy_count_t) -> task { co_return; }; + for(std::size_t i = 0; i < 4; ++i) + { + auto t = probe(destroy_count_t(down[i])); + conts[i].h = t.handle(); + t.release(); + } + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + strand s1{qe}; + strand s2{qe}; + s1.post(conts[0]); + s1.post(conts[1]); + s2.post(conts[2]); + s2.post(conts[3]); + BOOST_TEST_EQ(q.size(), 2u); + while(!q.empty()) + { + capy::abandon(q.front()); + q.pop(); + } + for(int d : down) + BOOST_TEST_EQ(d, 0); + + ctx.reset(); + for(int d : down) + BOOST_TEST_EQ(d, 1); + } + void testShutdownReleasesLiveStrand() { @@ -970,9 +1082,496 @@ struct strand_test (void)s; } + // A strand's first post racing its service's shutdown either queues + // an invoker the test then destroys, or is destroyed at once; the + // closed invoker cache is never reopened. + void + testFirstPostRacesShutdown() + { + struct locked_queue + { + std::mutex m; + std::vector v; + }; + struct locked_executor + { + locked_queue* q_; + test_io_context* ctx_; + + bool operator==(locked_executor const& o) const noexcept + { + return q_ == o.q_; + } + test_io_context& context() const noexcept { return *ctx_; } + void on_work_started() const noexcept {} + void on_work_finished() const noexcept {} + std::coroutine_handle<> dispatch(continuation& c) const + { + post(c); + return std::noop_coroutine(); + } + void post(continuation& c) const + { + std::lock_guard lk(q_->m); + q_->v.push_back(c); + } + }; + auto probe = [](destroy_count_t) -> task { co_return; }; + for(int i = 0; i < 1000; ++i) + { + int down = 0; + locked_queue q; + auto ctx = std::make_unique(); + locked_executor le{&q, ctx.get()}; + auto s = std::make_unique>(le); + auto t = probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + std::atomic go{0}; + std::thread a([&]{ + go.fetch_add(1); + while(go.load() < 2) + std::this_thread::yield(); + s->post(c); + }); + go.fetch_add(1); + while(go.load() < 2) + std::this_thread::yield(); + ctx->shutdown(); + a.join(); + for(auto& queued : q.v) + capy::abandon(queued); + q.v.clear(); + s.reset(); + ctx.reset(); + BOOST_TEST_EQ(down, 1); + } + } + + // An inner post that throws leaves the strand as it was: the caller + // still holds its continuation, and the next post runs. + void + testInnerPostThrows() + { + struct throwing_executor + { + std::queue* q_; + test_io_context* ctx_; + bool* fail_; + + bool operator==(throwing_executor const& o) const noexcept + { + return q_ == o.q_; + } + test_io_context& context() const noexcept { return *ctx_; } + void on_work_started() const noexcept {} + void on_work_finished() const noexcept {} + std::coroutine_handle<> dispatch(continuation& c) const + { + post(c); + return std::noop_coroutine(); + } + void post(continuation& c) const + { + if(*fail_) + throw std::runtime_error("post"); + q_->push(c); + } + }; + std::atomic ran{0}; + std::queue q; + bool fail = true; + auto held = make_counter_coro(ran); + auto ctx = std::make_unique(); + auto s = std::make_unique>( + throwing_executor{&q, ctx.get(), &fail}); + continuation c1{held.handle()}; + BOOST_TEST_THROWS(s->post(c1), std::runtime_error); + fail = false; + auto next = make_counter_coro(ran); + continuation c2{next.handle()}; + next.release(); + s->post(c2); + while(!q.empty()) + { + auto c = q.front(); + q.pop(); + c.h.resume(); + } + BOOST_TEST_EQ(ran.load(), 1); + s.reset(); + ctx.reset(); + } + + // Work queued behind a continuation whose invoker could not be + // posted still runs with no later post to restart the strand, or + // is given up if no invoker can be posted for it either. + void + testPostQueuedBehindThrowingPost() + { + auto attempt = [](int fails, int& ran, int& down) + { + std::atomic count{0}; + std::queue q; + std::function on_fail; + auto held = make_counter_coro(count); + auto behind = [](std::atomic* n, destroy_count_t) + -> counter_coro + { + ++*n; + co_return; + }(&count, destroy_count_t(down)); + continuation c1{held.handle()}; + continuation c2{behind.handle()}; + behind.release(); + auto ctx = std::make_unique(); + auto s = std::make_unique>( + reentrant_executor{&q, ctx.get(), &fails, &on_fail}); + on_fail = [&] { s->post(c2); }; + BOOST_TEST_THROWS(s->post(c1), std::runtime_error); + while(!q.empty()) + { + auto c = q.front(); + q.pop(); + c.h.resume(); + } + ran = count.load(); + s.reset(); + ctx.reset(); + }; + { + int ran = 0; + int down = 0; + attempt(1, ran, down); + BOOST_TEST_EQ(ran, 1); + BOOST_TEST_EQ(down, 1); + } + { + int ran = 0; + int down = 0; + attempt(2, ran, down); + BOOST_TEST_EQ(ran, 0); + BOOST_TEST_EQ(down, 1); + } + } + + // A post that throws after a shutdown took its continuation has + // nothing to hand back: the shutdown already destroyed it. + void + testPostThrowsAfterShutdownTookIt() + { + int down = 0; + int fails = 1; + std::queue q; + std::function on_fail; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto t = probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + auto ctx = std::make_unique(); + auto s = std::make_unique>( + reentrant_executor{&q, ctx.get(), &fails, &on_fail}); + on_fail = [&] { ctx->shutdown(); }; + bool threw = false; + try + { + s->post(c); + } + catch(std::runtime_error const&) + { + threw = true; + } + BOOST_TEST(!threw); + BOOST_TEST_EQ(down, 1); + BOOST_TEST(q.empty()); + s.reset(); + ctx.reset(); + } + + // When the retried invoker for work queued behind a failed post + // also fails after a shutdown took that work, the caller keeps its + // own continuation and the shutdown has destroyed the rest. + void + testRetryThrowsAfterShutdownTookFollowers() + { + int held_down = 0, behind_down = 0; + int fails = 2; + std::queue q; + std::function on_fail; + auto probe = [](destroy_count_t) -> task { co_return; }; + { + auto held = probe(destroy_count_t(held_down)); + auto behind = probe(destroy_count_t(behind_down)); + continuation c1{held.handle()}; + continuation c2{behind.handle()}; + behind.release(); + auto ctx = std::make_unique(); + auto s = std::make_unique>( + reentrant_executor{&q, ctx.get(), &fails, &on_fail}); + on_fail = [&] + { + s->post(c2); + on_fail = [&] { ctx->shutdown(); }; + }; + BOOST_TEST_THROWS(s->post(c1), std::runtime_error); + BOOST_TEST_EQ(held_down, 0); + BOOST_TEST_EQ(behind_down, 1); + BOOST_TEST(q.empty()); + s.reset(); + ctx.reset(); + } + BOOST_TEST_EQ(held_down, 1); + } + + // A strand made after its service shut down is closed from the + // start: what is posted to it is destroyed, and its handle can be + // dropped after the context is gone. + void + testStrandCreatedAfterShutdownIsClosed() + { + int down = 0; + std::queue q; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto t = probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + strand before{qe}; + ctx->shutdown(); + auto after = std::make_unique>(qe); + after->post(c); + BOOST_TEST_EQ(down, 1); + BOOST_TEST(q.empty()); + ctx.reset(); + after.reset(); + } + + // The same when the strand service itself is first created after + // the shutdown. + void + testStrandServiceCreatedAfterShutdown() + { + int down = 0; + std::queue q; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto t = probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + ctx->shutdown(); + auto s = std::make_unique>(qe); + s->post(c); + BOOST_TEST_EQ(down, 1); + BOOST_TEST(q.empty()); + ctx.reset(); + s.reset(); + } + + // An older service's destructor runs after the strand service is + // gone; a post from it is destroyed without touching that service. + void + testPostAfterServiceDestroyed() + { + static int down; + down = 0; + struct older_service : execution_context::service + { + std::queue q; + std::optional> s; + continuation c; + + explicit older_service(execution_context&) {} + + ~older_service() + { + auto probe = [](destroy_count_t) -> task + { + co_return; + }; + auto t = probe(destroy_count_t(down)); + c.h = t.handle(); + t.release(); + s->post(c); + } + + void shutdown() override {} + }; + { + test_io_context ctx; + auto& svc = ctx.make_service(); + queuing_executor qe(svc.q); + qe.ctx_ = &ctx; + svc.s.emplace(qe); + } + BOOST_TEST_EQ(down, 1); + } + + // A strand whose service has shut down abandons what is posted to + // it, since nothing will ever run it. + void + testPostAfterShutdownDestroyed() + { + int down = 0; + std::queue q; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto t = probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + auto ctx = std::make_unique(); + queuing_executor qe(q); + qe.ctx_ = ctx.get(); + strand s{qe}; + ctx->shutdown(); + s.post(c); + BOOST_TEST_EQ(down, 1); + BOOST_TEST(q.empty()); + } + + // Coroutines queued behind a drained invoker have no runner once + // the strand is dropped; the strand destroys them. + void + testQueuedCoroutinesDestroyedWithStrand() + { + int a_down = 0, b_down = 0; + std::queue q; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto ta = probe(destroy_count_t(a_down)); + auto tb = probe(destroy_count_t(b_down)); + continuation ca{ta.handle()}; + continuation cb{tb.handle()}; + ta.release(); + tb.release(); + { + strand s{queuing_executor(q)}; + s.post(ca); + s.post(cb); + BOOST_TEST_EQ(q.size(), 1u); + capy::abandon(q.front()); + q.pop(); + } + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(b_down, 1); + } + + // A frame's parameter holds the last strand handle, so tearing the + // frame down destroys a parented coroutine queued on the strand + // while the frame's own parent is still deferred. + void + testStrandParameterDestroysQueuedParentedCoroutine() + { + int pa_down = 0, pb_down = 0; + parked slot_q, slot_f; + std::queue q; + auto qa = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto pa = [qa](parked* s, destroy_count_t) + -> task + { + co_await qa(s); + }; + auto fb = [](strand, parked* s) + -> task + { + co_await park_awaitable{s}; + }; + auto pb = [fb]( + strand st, + parked* s, + destroy_count_t) -> task + { + co_await fb(std::move(st), s); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(pa(&slot_q, destroy_count_t(pa_down))); + BOOST_TEST(slot_q); + + continuation c{slot_q}; + { + strand s{queuing_executor(q)}; + s.post(c); + BOOST_TEST_EQ(q.size(), 1u); + capy::abandon(q.front()); + q.pop(); + run_async(ex)(pb(std::move(s), &slot_f, destroy_count_t(pb_down))); + } + BOOST_TEST(slot_f); + BOOST_TEST_EQ(pa_down, 0); + + capy::abandon(slot_f); + BOOST_TEST_EQ(pa_down, 1); + BOOST_TEST_EQ(pb_down, 1); + } + + // The last strand handle sits in a frame that an outer teardown + // destroys, so every queued coroutine and its parent are destroyed + // inside that teardown, however many there are. + void + testCascadeDropsLargeStrandBatch() + { + constexpr std::size_t count = 64; + int parents_down = 0, holder_down = 0; + std::array slots{}; + std::array conts{}; + parked slot_leaf; + std::queue q; + auto queued = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto parent = [queued](parked* s, destroy_count_t) + -> task + { + co_await queued(s); + }; + auto leaf = [](parked* s) -> task + { + co_await park_awaitable{s}; + }; + auto holder = [leaf]( + strand, + parked* s, + destroy_count_t) -> task + { + co_await leaf(s); + }; + int n = 0; + test_executor ex(n); + for(std::size_t i = 0; i < count; ++i) + run_async(ex)(parent(&slots[i], destroy_count_t(parents_down))); + + { + strand s{queuing_executor(q)}; + for(std::size_t i = 0; i < count; ++i) + { + conts[i] = slots[i]; + s.post(conts[i]); + } + BOOST_TEST_EQ(q.size(), 1u); + capy::abandon(q.front()); + q.pop(); + run_async(ex)(holder( + std::move(s), &slot_leaf, destroy_count_t(holder_down))); + } + BOOST_TEST(slot_leaf); + BOOST_TEST_EQ(parents_down, 0); + + capy::abandon(slot_leaf); + BOOST_TEST_EQ(parents_down, static_cast(count)); + BOOST_TEST_EQ(holder_down, 1); + } + void run() { + testShutdownDestroysQueuedAcrossStrands(); + testShutdownDestroysQueuedParentedChain(); testShutdownReleasesLiveStrand(); testConstruct(); testCopy(); @@ -995,6 +1594,18 @@ struct strand_test testSerialization(); testExitStrandAfterRun(); testAnyExecutor(); + testQueuedCoroutinesDestroyedWithStrand(); + testPostAfterShutdownDestroyed(); + testStrandCreatedAfterShutdownIsClosed(); + testFirstPostRacesShutdown(); + testInnerPostThrows(); + testPostQueuedBehindThrowingPost(); + testPostThrowsAfterShutdownTookIt(); + testRetryThrowsAfterShutdownTookFollowers(); + testStrandServiceCreatedAfterShutdown(); + testPostAfterServiceDestroyed(); + testStrandParameterDestroysQueuedParentedCoroutine(); + testCascadeDropsLargeStrandBatch(); } }; diff --git a/test/unit/ex/thread_pool.cpp b/test/unit/ex/thread_pool.cpp index 4ba750c08..a71308f61 100644 --- a/test/unit/ex/thread_pool.cpp +++ b/test/unit/ex/thread_pool.cpp @@ -16,11 +16,17 @@ #include #include #include +#include #include "test_helpers.hpp" #include #include +#include +#include +#include +#include +#include #include #include @@ -138,6 +144,134 @@ name_checker check_thread_name(name_check_result& result, char const* prefix) } #endif +// Posts during shutdown, as a primitive owned by a frame that a +// service's shutdown destroys would. +struct late_poster : execution_context::service +{ + thread_pool::executor_type ex_; + continuation* c_; + + late_poster( + execution_context&, + thread_pool::executor_type ex, + continuation& c) + : ex_(ex) + , c_(&c) + { + } + + void shutdown() override + { + ex_.post(*c_); + } +}; + +// Posts from its destructor, after the pool's services shut down. +struct dtor_poster : execution_context::service +{ + thread_pool::executor_type ex_; + continuation* c_; + + dtor_poster( + execution_context&, + thread_pool::executor_type ex, + continuation& c) + : ex_(ex) + , c_(&c) + { + } + + ~dtor_poster() + { + ex_.post(*c_); + } + + void shutdown() override {} +}; + +// Starts posting from another thread when the pool's services shut +// down, while the pool drains, and joins that thread when destroyed. +struct shutdown_poster : execution_context::service +{ + thread_pool::executor_type ex_; + std::vector* cs_; + std::thread t_; + + shutdown_poster( + execution_context&, + thread_pool::executor_type ex, + std::vector& cs) + : ex_(ex) + , cs_(&cs) + { + } + + ~shutdown_poster() + { + t_.join(); + } + + void shutdown() override + { + t_ = std::thread([this] + { + for(auto& c : *cs_) + ex_.post(c); + }); + } +}; + +// A second dtor_poster, so one can be created while the first exists. +struct late_dtor_poster : dtor_poster +{ + using dtor_poster::dtor_poster; +}; + +// Creates a late_dtor_poster on the pool when destroyed. +struct poster_maker +{ + thread_pool* pool; + continuation* c; + + poster_maker(thread_pool& p, continuation& cont) noexcept + : pool(&p) + , c(&cont) + { + } + + poster_maker(poster_maker&& o) noexcept + : pool(std::exchange(o.pool, nullptr)) + , c(o.c) + { + } + + ~poster_maker() + { + if(pool) + pool->make_service(pool->get_executor(), *c); + } +}; + +// Parks the awaiting coroutine in a slot and signals that it did. +struct signal_park +{ + capy::parked* slot; + std::binary_semaphore* parked; + + bool await_ready() const noexcept { return false; } + + std::coroutine_handle<> + await_suspend(std::coroutine_handle<> h, io_env const* env) noexcept + { + slot->h = h; + slot->owner = env->owner; + parked->release(); + return std::noop_coroutine(); + } + + void await_resume() const noexcept {} +}; + } // namespace struct thread_pool_test @@ -728,6 +862,287 @@ struct thread_pool_test worker.join(); } + // A first post racing join() either starts threads join() then + // joins, or starts none; the destructor never meets a running one. + // A thread left running fails the test by terminating the process. + void + testJoinRacesFirstPost() + { + for(int i = 0; i < 200; ++i) + { + continuation c{std::noop_coroutine()}; + thread_pool pool(4); + std::atomic go{false}; + std::thread t([&]{ + while(!go.load()) + { + } + pool.get_executor().post(c); + }); + go.store(true); + pool.join(); + t.join(); + } + } + + // A stop() on another thread is done with the pool before the join() + // it releases can return and let the pool be destroyed. Touching the + // destroyed pool shows only under a sanitizer. + void + testStopRacesJoinAndDestroy() + { + for(int i = 0; i < 300; ++i) + { + std::thread t; + { + auto pool = std::make_unique(1); + auto ex = pool->get_executor(); + ex.on_work_started(); + std::atomic go{false}; + t = std::thread([p = pool.get(), &go]{ + while(!go.load()) + { + } + p->stop(); + }); + go.store(true); + pool->join(); + pool.reset(); + } + t.join(); + } + } + + // join() waiting on counted work must not stop the first post from + // starting the threads that work needs. Nothing shows when join() + // is entered, so only iterations where it is already waiting when + // the post arrives exercise that. + void + testJoinWaitingThenFirstPost() + { + for(int i = 0; i < 20; ++i) + { + thread_pool pool(2); + auto ex = pool.get_executor(); + auto guard = std::make_unique< + work_guard>(ex); + std::atomic joining{false}; + std::mutex m; + std::condition_variable cv; + bool joined = false; + std::thread j([&]{ + joining.store(true); + pool.join(); + std::lock_guard lk(m); + joined = true; + cv.notify_all(); + }); + while(!joining.load()) + std::this_thread::yield(); + std::atomic ran{false}; + auto t = [](std::atomic& r) -> task + { + r.store(true); + co_return; + }; + run_async(ex)(t(ran)); + guard.reset(); + bool ok; + { + // Only a watchdog: a broken pool never returns from join(). + std::unique_lock lk(m); + ok = cv.wait_for(lk, std::chrono::seconds(30), + [&]{ return joined; }); + } + BOOST_TEST(ok); + BOOST_TEST(ran.load()); + if(!ok) + pool.stop(); + j.join(); + } + } + + // The last work count released from a foreign thread is done with + // the pool before join() can see it reach zero. Touching the + // destroyed pool shows only under a sanitizer. + void + testForeignWorkFinishedRacesJoin() + { + for(int i = 0; i < 500; ++i) + { + std::thread t; + { + auto pool = std::make_unique(1); + auto ex = pool->get_executor(); + ex.on_work_started(); + std::atomic go{false}; + t = std::thread([ex, &go]{ + while(!go.load()) + { + } + ex.on_work_finished(); + }); + go.store(true); + pool->join(); + pool.reset(); + } + t.join(); + } + } + + // A chain whose root runs on a live pool, torn down from another + // thread, comes down once with every frame freed, and the pool's + // work is released so join() returns. + void + testTeardownFromAnotherThreadWithLiveRoot() + { + alloc_counts st; + int child_down = 0, parent_down = 0, handler_down = 0; + parked slot; + std::binary_semaphore parked{0}; + thread_pool pool(1); + auto child = [&](destroy_count_t) -> task + { + co_await signal_park{&slot, &parked}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await child(destroy_count_t(child_down)); + }; + run_async(pool.get_executor(), counting_allocator(st), + [d = destroy_count_t(handler_down)] { BOOST_TEST(false); }, + [](std::exception_ptr) { BOOST_TEST(false); })( + parent(destroy_count_t(parent_down))); + // Bounds a hang, not a timing assertion. + if(!BOOST_TEST(parked.try_acquire_for(std::chrono::seconds(30)))) + return; + + capy::abandon(slot); + pool.join(); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST_EQ(handler_down, 1); + BOOST_TEST(st.allocs.load() > 0); + BOOST_TEST(st.balanced()); + } + + // A frame destroyed by the pool's last drain may create a service + // whose destructor posts again; that work is destroyed too. + void + testServiceCreatedByLastDrainDestroyed() + { + int down = 0; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto maker = [](poster_maker) -> task { co_return; }; + auto p = probe(destroy_count_t(down)); + continuation probe_c{p.handle()}; + p.release(); + continuation maker_c; + { + thread_pool pool(1); + auto m = maker(poster_maker(pool, probe_c)); + maker_c.h = m.handle(); + m.release(); + pool.make_service(pool.get_executor(), maker_c); + } + BOOST_TEST_EQ(down, 1); + } + + // A started chain whose leaf is queued when the pool is destroyed + // comes down there, on another thread than its child was made on, + // with every frame freed. + void + testDestroyDrainsParentedChain() + { + alloc_counts st; + int child_down = 0, parent_down = 0, handler_down = 0; + parked slot; + std::binary_semaphore parked{0}; + continuation c; + { + thread_pool pool(1); + auto child = [&](destroy_count_t) -> task + { + co_await signal_park{&slot, &parked}; + }; + auto parent = [&](destroy_count_t) -> task + { + co_await child(destroy_count_t(child_down)); + }; + run_async(pool.get_executor(), counting_allocator(st), + [d = destroy_count_t(handler_down)] { BOOST_TEST(false); }, + [](std::exception_ptr) { BOOST_TEST(false); })( + parent(destroy_count_t(parent_down))); + // Bounds a hang, not a timing assertion. + if(!BOOST_TEST(parked.try_acquire_for(std::chrono::seconds(30)))) + return; + c = slot; + pool.make_service(pool.get_executor(), c); + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST_EQ(handler_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // Work posted while the pool's services shut down is destroyed with + // the pool rather than left queued. + void + testPostDuringShutdownDestroyed() + { + int down = 0; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto t = probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + { + thread_pool pool(1); + pool.make_service(pool.get_executor(), c); + } + BOOST_TEST_EQ(down, 1); + } + + // Work posted while the pool's services are destroyed is destroyed + // with the pool. + void + testPostFromServiceDestructorDestroyed() + { + int down = 0; + auto probe = [](destroy_count_t) -> task { co_return; }; + auto t = probe(destroy_count_t(down)); + continuation c{t.handle()}; + t.release(); + { + thread_pool pool(1); + pool.make_service(pool.get_executor(), c); + } + BOOST_TEST_EQ(down, 1); + } + + // Work posted from another thread while the pool drains is neither + // lost nor torn from the queue. + void + testPostDuringDrain() + { + constexpr int n = 2000; + int down = 0; + auto probe = [](destroy_count_t) -> task { co_return; }; + std::vector cs; + cs.reserve(n); + for(int i = 0; i < n; ++i) + { + auto t = probe(destroy_count_t(down)); + cs.push_back(continuation{t.handle()}); + t.release(); + } + { + thread_pool pool(1); + pool.make_service(pool.get_executor(), cs); + } + BOOST_TEST_EQ(down, n); + } + void run() { @@ -758,6 +1173,16 @@ struct thread_pool_test testWorkGuardKeepsPoolAlive(); testJoinWithRunAsync(); testForeignPostAfterJoin(); + testPostDuringShutdownDestroyed(); + testPostFromServiceDestructorDestroyed(); + testPostDuringDrain(); + testDestroyDrainsParentedChain(); + testServiceCreatedByLastDrainDestroyed(); + testTeardownFromAnotherThreadWithLiveRoot(); + testJoinRacesFirstPost(); + testJoinWaitingThenFirstPost(); + testStopRacesJoinAndDestroy(); + testForeignWorkFinishedRacesJoin(); } }; diff --git a/test/unit/io/any_read_stream.cpp b/test/unit/io/any_read_stream.cpp index 6671f7025..7d4772516 100644 --- a/test/unit/io/any_read_stream.cpp +++ b/test/unit/io/any_read_stream.cpp @@ -11,6 +11,8 @@ #include #include +#include +#include #include #include #include @@ -18,14 +20,17 @@ #include #include +#include "test/unit/io/owned_parking_stream.hpp" #include "test/unit/test_helpers.hpp" #include #include #include +#include #include #include +#include #include namespace boost { @@ -49,6 +54,35 @@ struct pending_read_awaitable { return {std::error_code(), 0}; } }; +// Parks the awaiting coroutine for the test to resume or abandon, +// and counts how often the awaitable is destroyed. +struct parking_read_awaitable +{ + parked* slot_; + int* down_; + parking_read_awaitable(parked* s, int* d) : slot_(s), down_(d) {} + parking_read_awaitable(parking_read_awaitable&& o) noexcept + : slot_(o.slot_), down_(std::exchange(o.down_, nullptr)) {} + ~parking_read_awaitable() { if(down_) ++(*down_); } + bool await_ready() const noexcept { return false; } + std::coroutine_handle<> + await_suspend(std::coroutine_handle<> h, io_env const* env) noexcept + { + return park_awaitable{slot_}.await_suspend(h, env); + } + io_result await_resume() + { return {std::error_code(), 0}; } +}; + +struct parking_read_stream +{ + parked* slot_; + int* down_; + parking_read_awaitable read_some( + MutableBufferSequence auto) + { return parking_read_awaitable{slot_, down_}; } +}; + struct pending_read_stream { int* counter_; @@ -556,6 +590,26 @@ class any_read_stream_test BOOST_TEST_EQ(destroyed, 1); } + // A started read moved to another awaitable takes its claim with + // it, so the wrapper's teardown leaves the moved-to awaitable owning + // nothing. + void + testMoveStartedAwaitable() + { + int destroyed = 0; + pending_read_stream ps{&destroyed}; + { + auto s = std::make_unique(&ps); + char data[1]; + auto aw = s->read_some(mutable_buffer(data, 1)); + BOOST_TEST(!aw.await_ready()); + auto moved = std::move(aw); + s.reset(); + BOOST_TEST_EQ(destroyed, 1); + } + BOOST_TEST_EQ(destroyed, 1); + } + void testMoveAssignWithActiveAwaitable() { @@ -622,12 +676,128 @@ class any_read_stream_test BOOST_TEST_EQ(result, 7u); } + // A chain unwound while parked in a type-erased read destroys the + // read it waited on, as it would the stream's own awaitable. + void + testUnwindDestroysInnerRead() + { + parked slot; + int down = 0; + parking_read_stream ps{&slot, &down}; + any_read_stream s(&ps); + int dc = 0; + test_executor ex(dc); + run_async(ex)([](any_read_stream& s) -> task + { + char c = 0; + std::ignore = co_await s.read_some(mutable_buffer(&c, 1)); + }(s)); + BOOST_TEST(static_cast(slot)); + { + // Another read, made but never awaited, owns nothing. + char d = 0; + auto unused = s.read_some(mutable_buffer(&d, 1)); + } + BOOST_TEST_EQ(down, 0); + capy::abandon(slot); + BOOST_TEST_EQ(down, 1); + } + + // A wrapper destroyed while a read is parked on it has already + // destroyed that read; unwinding the chain afterwards must not + // touch the wrapper. + void + testUnwindAfterWrapperDestroyed() + { + parked slot; + int down = 0; + parking_read_stream ps{&slot, &down}; + auto s = std::make_unique(&ps); + int dc = 0; + test_executor ex(dc); + run_async(ex)([](any_read_stream& s) -> task + { + char c = 0; + std::ignore = co_await s.read_some(mutable_buffer(&c, 1)); + }(*s)); + BOOST_TEST(static_cast(slot)); + s.reset(); + BOOST_TEST_EQ(down, 1); + capy::abandon(slot); + BOOST_TEST_EQ(down, 1); + } + + // A read unwound after its stream was assigned to leaves the + // read the stream started since then alone. + void + testUnwindAfterStreamAssigned() + { + parked first; + parked second; + int down = 0; + parking_read_stream ps1{&first, &down}; + parking_read_stream ps2{&second, &down}; + any_read_stream s(&ps1); + int dc = 0; + test_executor ex(dc); + auto op = [](any_read_stream& s) -> task + { + char c = 0; + std::ignore = co_await s.read_some(mutable_buffer(&c, 1)); + }; + run_async(ex)(op(s)); + BOOST_TEST(static_cast(first)); + s = any_read_stream(&ps2); + BOOST_TEST_EQ(down, 1); + run_async(ex)(op(s)); + BOOST_TEST(static_cast(second)); + capy::abandon(first); + BOOST_TEST_EQ(down, 1); + capy::abandon(second); + BOOST_TEST_EQ(down, 2); + } + + // A wrapper destroyed or assigned to while a read is parked on the + // stream it owns ends that read before destroying the stream. + void + testOwningStreamEndsWithReadParked(bool assign) + { + owned_parking_stream::state st; + owned_parking_stream::state other; + auto s = std::make_unique(owned_parking_stream(st)); + int dc = 0; + test_executor ex(dc); + run_async(ex)([](any_read_stream& s) -> task + { + char c = 0; + std::ignore = co_await s.read_some(mutable_buffer(&c, 1)); + }(*s)); + BOOST_TEST(static_cast(st.slot)); + BOOST_TEST(st.alive); + + if(assign) + *s = any_read_stream(owned_parking_stream(other)); + else + s.reset(); + BOOST_TEST(!st.alive); + BOOST_TEST_EQ(st.ops_down, 1); + BOOST_TEST_EQ(st.late, 0); + + capy::abandon(st.slot); + BOOST_TEST_EQ(st.ops_down, 1); + } + void run() { + testOwningStreamEndsWithReadParked(false); + testOwningStreamEndsWithReadParked(true); testConstruct(); testMove(); testReadSome(); + testUnwindDestroysInnerRead(); + testUnwindAfterStreamAssigned(); + testUnwindAfterWrapperDestroyed(); testReadSomePartial(); testReadSomeMultiple(); testReadSomeEof(); @@ -643,6 +813,7 @@ class any_read_stream_test testReadyStreamSkipsSuspension(); testDestroyWithActiveAwaitable(); testMoveAssignWithActiveAwaitable(); + testMoveStartedAwaitable(); testMoveAssignOwning(); testConstructThrows(); testReadSomeSuspends(); diff --git a/test/unit/io/any_stream.cpp b/test/unit/io/any_stream.cpp index ab788a30c..6974ba29f 100644 --- a/test/unit/io/any_stream.cpp +++ b/test/unit/io/any_stream.cpp @@ -13,14 +13,19 @@ #include #include #include +#include #include +#include #include #include +#include "test/unit/io/owned_parking_stream.hpp" #include "test/unit/test_helpers.hpp" +#include #include #include +#include namespace boost { namespace capy { @@ -290,9 +295,47 @@ class any_stream_test BOOST_TEST(r.success); } + // A wrapper destroyed or assigned to while a read or write is + // parked on the stream it owns ends that operation before + // destroying the stream. + void + testOwningStreamEndsWithOpParked(bool write, bool assign) + { + owned_parking_stream::state st; + owned_parking_stream::state other; + auto s = std::make_unique(owned_parking_stream(st)); + int dc = 0; + test_executor ex(dc); + run_async(ex)([](any_stream& s, bool write) -> task + { + char c = 0; + if(write) + std::ignore = co_await s.write_some(const_buffer(&c, 1)); + else + std::ignore = co_await s.read_some(mutable_buffer(&c, 1)); + }(*s, write)); + BOOST_TEST(static_cast(st.slot)); + BOOST_TEST(st.alive); + + if(assign) + *s = any_stream(owned_parking_stream(other)); + else + s.reset(); + BOOST_TEST(!st.alive); + BOOST_TEST_EQ(st.ops_down, 1); + BOOST_TEST_EQ(st.late, 0); + + capy::abandon(st.slot); + BOOST_TEST_EQ(st.ops_down, 1); + } + void run() { + testOwningStreamEndsWithOpParked(false, false); + testOwningStreamEndsWithOpParked(false, true); + testOwningStreamEndsWithOpParked(true, false); + testOwningStreamEndsWithOpParked(true, true); testConstruct(); testMove(); testMoveAssignOwning(); diff --git a/test/unit/io/any_write_stream.cpp b/test/unit/io/any_write_stream.cpp index 1d0884681..d07dd7e48 100644 --- a/test/unit/io/any_write_stream.cpp +++ b/test/unit/io/any_write_stream.cpp @@ -11,6 +11,8 @@ #include #include +#include +#include #include #include #include @@ -24,6 +26,7 @@ #include #include +#include #include #include #include @@ -49,6 +52,35 @@ struct pending_write_awaitable { return {std::error_code(), 0}; } }; +// Parks the awaiting coroutine for the test to resume or abandon, +// and counts how often the awaitable is destroyed. +struct parking_write_awaitable +{ + parked* slot_; + int* down_; + parking_write_awaitable(parked* s, int* d) : slot_(s), down_(d) {} + parking_write_awaitable(parking_write_awaitable&& o) noexcept + : slot_(o.slot_), down_(std::exchange(o.down_, nullptr)) {} + ~parking_write_awaitable() { if(down_) ++(*down_); } + bool await_ready() const noexcept { return false; } + std::coroutine_handle<> + await_suspend(std::coroutine_handle<> h, io_env const* env) noexcept + { + return park_awaitable{slot_}.await_suspend(h, env); + } + io_result await_resume() + { return {std::error_code(), 0}; } +}; + +struct parking_write_stream +{ + parked* slot_; + int* down_; + parking_write_awaitable write_some( + ConstBufferSequence auto) + { return parking_write_awaitable{slot_, down_}; } +}; + struct pending_write_stream { int* counter_; @@ -500,6 +532,26 @@ class any_write_stream_test BOOST_TEST_EQ(destroyed, 1); } + // A started write moved to another awaitable takes its claim with + // it, so the wrapper's teardown leaves the moved-to awaitable owning + // nothing. + void + testMoveStartedAwaitable() + { + int destroyed = 0; + pending_write_stream ps{&destroyed}; + { + auto s = std::make_unique(&ps); + char const data[] = "x"; + auto aw = s->write_some(const_buffer(data, 1)); + BOOST_TEST(!aw.await_ready()); + auto moved = std::move(aw); + s.reset(); + BOOST_TEST_EQ(destroyed, 1); + } + BOOST_TEST_EQ(destroyed, 1); + } + void testMoveAssignWithActiveAwaitable() { @@ -522,12 +574,96 @@ class any_write_stream_test } } + // A chain unwound while parked in a type-erased write destroys the + // write it waited on, as it would the stream's own awaitable. + void + testUnwindDestroysInnerWrite() + { + parked slot; + int down = 0; + parking_write_stream ps{&slot, &down}; + any_write_stream s(&ps); + int dc = 0; + test_executor ex(dc); + run_async(ex)([](any_write_stream& s) -> task + { + char c = 0; + std::ignore = co_await s.write_some(make_buffer(&c, 1)); + }(s)); + BOOST_TEST(static_cast(slot)); + { + // Another write, made but never awaited, owns nothing. + char d = 0; + auto unused = s.write_some(make_buffer(&d, 1)); + } + BOOST_TEST_EQ(down, 0); + capy::abandon(slot); + BOOST_TEST_EQ(down, 1); + } + + // A wrapper destroyed while a write is parked on it has already + // destroyed that write; unwinding the chain afterwards must not + // touch the wrapper. + void + testUnwindAfterWrapperDestroyed() + { + parked slot; + int down = 0; + parking_write_stream ps{&slot, &down}; + auto s = std::make_unique(&ps); + int dc = 0; + test_executor ex(dc); + run_async(ex)([](any_write_stream& s) -> task + { + char c = 0; + std::ignore = co_await s.write_some(make_buffer(&c, 1)); + }(*s)); + BOOST_TEST(static_cast(slot)); + s.reset(); + BOOST_TEST_EQ(down, 1); + capy::abandon(slot); + BOOST_TEST_EQ(down, 1); + } + + // A write unwound after its stream was assigned to leaves the + // write the stream started since then alone. + void + testUnwindAfterStreamAssigned() + { + parked first; + parked second; + int down = 0; + parking_write_stream ps1{&first, &down}; + parking_write_stream ps2{&second, &down}; + any_write_stream s(&ps1); + int dc = 0; + test_executor ex(dc); + auto op = [](any_write_stream& s) -> task + { + char c = 0; + std::ignore = co_await s.write_some(make_buffer(&c, 1)); + }; + run_async(ex)(op(s)); + BOOST_TEST(static_cast(first)); + s = any_write_stream(&ps2); + BOOST_TEST_EQ(down, 1); + run_async(ex)(op(s)); + BOOST_TEST(static_cast(second)); + capy::abandon(first); + BOOST_TEST_EQ(down, 1); + capy::abandon(second); + BOOST_TEST_EQ(down, 2); + } + void run() { testConstruct(); testMove(); testWriteSome(); + testUnwindDestroysInnerWrite(); + testUnwindAfterStreamAssigned(); + testUnwindAfterWrapperDestroyed(); testWriteSomePartial(); testWriteSomeMultiple(); testWriteSomeBufferSequence(); @@ -541,6 +677,7 @@ class any_write_stream_test testReadyStreamSkipsSuspension(); testDestroyWithActiveAwaitable(); testMoveAssignWithActiveAwaitable(); + testMoveStartedAwaitable(); testMoveAssignOwning(); testConstructThrows(); } diff --git a/test/unit/io/owned_parking_stream.hpp b/test/unit/io/owned_parking_stream.hpp new file mode 100644 index 000000000..be750967e --- /dev/null +++ b/test/unit/io/owned_parking_stream.hpp @@ -0,0 +1,114 @@ +// +// Copyright (c) 2026 Steve Gerbino +// +// Distributed under the Boost Software License, Version 1.0. (See accompanying +// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) +// +// Official repository: https://github.com/cppalliance/capy +// + +#ifndef BOOST_CAPY_TEST_OWNED_PARKING_STREAM_HPP +#define BOOST_CAPY_TEST_OWNED_PARKING_STREAM_HPP + +#include +#include + +#include "test/unit/test_helpers.hpp" + +#include +#include +#include +#include + +namespace boost { +namespace capy { + +/** A stream for a type-erased wrapper to own. + + Its reads and writes park the awaiting coroutine for the test to + resume or abandon. Each operation records, when it is destroyed, + whether the stream it came from was already gone. +*/ +struct owned_parking_stream +{ + /// What the stream and its operations report to the test. + struct state + { + parked slot; + int ops_down = 0; + int late = 0; + bool alive = false; + }; + + struct op + { + state* st_; + + ~op() + { + if(st_) + { + ++st_->ops_down; + if(!st_->alive) + ++st_->late; + } + } + + explicit op(state* st) noexcept + : st_(st) + { + } + + op(op&& o) noexcept + : st_(std::exchange(o.st_, nullptr)) + { + } + + bool await_ready() const noexcept { return false; } + + std::coroutine_handle<> + await_suspend(std::coroutine_handle<> h, io_env const* env) noexcept + { + return park_awaitable{&st_->slot}.await_suspend(h, env); + } + + io_result await_resume() noexcept + { + return {std::error_code(), 0}; + } + }; + + state* st_; + + ~owned_parking_stream() + { + if(st_) + st_->alive = false; + } + + explicit owned_parking_stream(state& st) noexcept + : st_(&st) + { + st.alive = true; + } + + owned_parking_stream(owned_parking_stream&& o) noexcept + : st_(std::exchange(o.st_, nullptr)) + { + } + + op read_some(MutableBufferSequence auto) + { + return op{st_}; + } + + op write_some(ConstBufferSequence auto) + { + return op{st_}; + } +}; + +} // capy +} // boost + +#endif diff --git a/test/unit/io/write_now.cpp b/test/unit/io/write_now.cpp index f9ba3db05..08be5ce2c 100644 --- a/test/unit/io/write_now.cpp +++ b/test/unit/io/write_now.cpp @@ -12,17 +12,22 @@ #include #include +#include +#include #include +#include #include #include #include +#include "test/unit/death_test.hpp" #include "test/unit/test_helpers.hpp" #include #include #include #include +#include namespace boost { namespace capy { @@ -46,6 +51,36 @@ struct suspending_write_stream { return {}; } }; +// A stream whose write_some parks the writer for the test. +struct parking_stream +{ + parked* slot; + + struct write_awaitable + { + parked* slot; + bool await_ready() const noexcept { return false; } + std::coroutine_handle<> + await_suspend(std::coroutine_handle<> h, io_env const* env) noexcept + { + slot->h = h; + slot->owner = env->owner; + return std::noop_coroutine(); + } + io_result await_resume() const noexcept + { + return {std::error_code(), 1}; + } + }; + + template + write_awaitable write_some(Buffers const&) + { + return {slot}; + } +}; +static_assert(WriteStream); + class write_now_test { public: @@ -316,9 +351,94 @@ class write_now_test BOOST_TEST(true); } + // Tearing down a parked write destroys the awaiting chain once, + // including the write_now that holds the op's frame memory. + void + testTeardownOfParkedWriteDestroysChainOnce() + { + alloc_counts st; + int parent_down = 0; + parked slot; + parking_stream s{&slot}; + auto parent = [&](destroy_count_t) -> task + { + write_now wn(s); + char c = 'x'; + std::ignore = co_await wn(const_buffer(&c, 1)); + }; + int n = 0; + test_executor ex(n); + run_async(ex, counting_allocator(st))( + parent(destroy_count_t(parent_down))); + BOOST_TEST(slot); + capy::abandon(slot); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 1); + BOOST_TEST(st.balanced()); + } + +#if !defined(_WIN32) + // Destroying the parent from above while its write is parked + // destroys the write's lent frame from above, which terminates. + void + testDestroyParentAwaitingWriteTerminates() + { + BOOST_TEST(terminates([] { + parked slot; + parking_stream s{&slot}; + auto parent = [&]() -> task + { + write_now wn(s); + char c = 'x'; + std::ignore = co_await wn(const_buffer(&c, 1)); + }; + int n = 0; + test_executor caller(n); + io_env env{executor_ref(caller), {}, nullptr}; + // Started by hand, not lent, so this destroys it from above. + auto t = parent(); + t.handle().promise().set_environment(&env); + t.handle().resume(); + })); + } +#endif + + // A teardown on the second write tears down a reused cached frame. + void + testTeardownOfReusedFrame() + { + int parent_down = 0; + std::size_t first = 0; + parked slot; + parking_stream s{&slot}; + auto parent = [&](destroy_count_t) -> task + { + write_now wn(s); + char c = 'x'; + auto [ec, n] = co_await wn(const_buffer(&c, 1)); + first = n; + std::ignore = co_await wn(const_buffer(&c, 1)); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + BOOST_TEST(slot); + std::exchange(slot, parked{}).resume(); + BOOST_TEST_EQ(first, 1u); + BOOST_TEST_EQ(parent_down, 0); + BOOST_TEST(slot); + capy::abandon(slot); + BOOST_TEST_EQ(parent_down, 1); + } + void run() { + testTeardownOfParkedWriteDestroysChainOnce(); +#if !defined(_WIN32) + testDestroyParentAwaitingWriteTerminates(); +#endif + testTeardownOfReusedFrame(); testFrameRealloc(); testSuspendingWrite(); testSingleBuffer(); diff --git a/test/unit/quitter.cpp b/test/unit/quitter.cpp index d15a67fe9..183237f1d 100644 --- a/test/unit/quitter.cpp +++ b/test/unit/quitter.cpp @@ -12,6 +12,7 @@ #include #include +#include #include #include #include @@ -24,8 +25,11 @@ #include #include +#include "death_test.hpp" #include "test_helpers.hpp" +#include + #include #include #include @@ -33,6 +37,7 @@ #include #include #include +#include #include namespace boost { @@ -45,6 +50,103 @@ static_assert(IoRunnable>); struct quitter_test { + // Assigning over a quitter that is not awaited destroys the frame + // it held. + void testMoveAssignDestroysHeldFrame() + { + int a_down = 0, b_down = 0; + auto body = [](destroy_count_t) -> quitter { co_return; }; + { + auto t = body(destroy_count_t(a_down)); + t = body(destroy_count_t(b_down)); + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(b_down, 0); + } + BOOST_TEST_EQ(b_down, 1); + } + +#if !defined(_WIN32) + void testDestroyLentTaskTerminates() + { + BOOST_TEST(terminates([] { + parked slot; + int n = 0; + test_executor ex(n); + io_env env{executor_ref(ex), {}, nullptr}; + auto t = [](parked* s) -> quitter { + co_await park_awaitable{s}; + }(&slot); + lend(t, env); + // destructor runs with the frame lent and suspended + })); + } + + void testMoveAssignOverLentTaskTerminates() + { + BOOST_TEST(terminates([] { + parked slot; + int n = 0; + test_executor ex(n); + io_env env{executor_ref(ex), {}, nullptr}; + auto body = [](parked* s) -> quitter { + co_await park_awaitable{s}; + }; + auto t = body(&slot); + lend(t, env); + t = body(&slot); + std::_Exit(0); // must not be reached + })); + } + + void testDestroyUnstartedOrFinishedTaskIsFine() + { + BOOST_TEST(exits_cleanly([] { + auto body = []() -> quitter { co_return; }; + { auto t = body(); } // never started + int n = 0; + test_executor ex(n); + io_env env{executor_ref(ex), {}, nullptr}; + auto t = body(); + lend(t, env); // runs to final_suspend + return t.handle().done(); + })); // finished frame destroyed + } +#endif + + // A quitter in the middle of a chain comes down with it when the + // leaf below it is torn down. + void testTeardownThroughQuitter() + { + alloc_counts st; + int leaf_down = 0, mid_down = 0, root_down = 0; + parked slot; + { + auto leaf = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto mid = [&](destroy_count_t) -> quitter + { + co_await leaf(&slot, destroy_count_t(leaf_down)); + }; + auto root = [&](destroy_count_t) -> task + { + co_await mid(destroy_count_t(mid_down)); + }; + int n = 0; + test_executor ex(n); + run_async(ex, counting_allocator(st))( + root(destroy_count_t(root_down))); + BOOST_TEST(slot); + capy::abandon(slot); + } + BOOST_TEST_EQ(leaf_down, 1); + BOOST_TEST_EQ(mid_down, 1); + BOOST_TEST_EQ(root_down, 1); + BOOST_TEST(st.balanced()); + } + //---------------------------------------------------------- // 1. Normal completion — quitter returns a value //---------------------------------------------------------- @@ -861,6 +963,13 @@ struct quitter_test void run() { + testMoveAssignDestroysHeldFrame(); +#if !defined(_WIN32) + testDestroyLentTaskTerminates(); + testMoveAssignOverLentTaskTerminates(); + testDestroyUnstartedOrFinishedTaskIsFine(); +#endif + testTeardownThroughQuitter(); testNormalCompletion(); testVoidCompletion(); testExceptionPropagation(); diff --git a/test/unit/task.cpp b/test/unit/task.cpp index a2200f03f..a5fad28d7 100644 --- a/test/unit/task.cpp +++ b/test/unit/task.cpp @@ -10,17 +10,24 @@ // Test that header file is self-contained. #include +#include #include +#include #include #include #include +#include "death_test.hpp" #include "test_helpers.hpp" +#include + #include #include +#include #include #include +#include #include #include #include @@ -91,8 +98,86 @@ struct tracking_executor static_assert(Executor); +namespace { + +// Parks at the bottom of a chain `depth` frames above the leaf. +task +nest_chain( + int depth, parked* slot, int* down, destroy_count_t) +{ + if(depth == 0) + co_await park_awaitable{slot}; + else + co_await nest_chain(depth - 1, slot, down, destroy_count_t(*down)); +} + +} // namespace + struct task_test { + // Assigning over a task that is not awaited destroys the frame + // it held. + void testMoveAssignDestroysHeldFrame() + { + int a_down = 0, b_down = 0; + auto body = [](destroy_count_t) -> task { co_return; }; + { + auto t = body(destroy_count_t(a_down)); + t = body(destroy_count_t(b_down)); + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(b_down, 0); + } + BOOST_TEST_EQ(b_down, 1); + } + +#if !defined(_WIN32) + void testDestroyLentTaskTerminates() + { + BOOST_TEST(terminates([] { + parked slot; + int n = 0; + test_executor ex(n); + io_env env{executor_ref(ex), {}, nullptr}; + auto t = [](parked* s) -> task { + co_await park_awaitable{s}; + }(&slot); + lend(t, env); + // destructor runs with the frame lent and suspended + })); + } + + void testMoveAssignOverLentTaskTerminates() + { + BOOST_TEST(terminates([] { + parked slot; + int n = 0; + test_executor ex(n); + io_env env{executor_ref(ex), {}, nullptr}; + auto body = [](parked* s) -> task { + co_await park_awaitable{s}; + }; + auto t = body(&slot); + lend(t, env); + t = body(&slot); + std::_Exit(0); // must not be reached + })); + } + + void testDestroyUnstartedOrFinishedTaskIsFine() + { + BOOST_TEST(exits_cleanly([] { + auto body = []() -> task { co_return; }; + { auto t = body(); } // never started + int n = 0; + test_executor ex(n); + io_env env{executor_ref(ex), {}, nullptr}; + auto t = body(); + lend(t, env); // runs to final_suspend + return t.handle().done(); + })); // finished frame destroyed + } +#endif + static task returns_int() { @@ -1217,6 +1302,286 @@ struct task_test BOOST_TEST(td2 == true); } + // Destroying a coroutine that a parent awaits destroys the parent + // too, and the parent's task must not destroy it a second time. + void testDestroyAwaitedChild() + { + int child_down = 0, parent_down = 0, handler_down = 0; + { + auto child = [](destroy_count_t) -> capy::task + { + co_await self_destroy_awaitable{}; + }; + auto parent = [&](destroy_count_t) -> capy::task + { + co_await child(destroy_count_t(child_down)); + }; + int dispatch_count = 0; + test_executor ex(dispatch_count); + run_async(ex, + [t = destroy_count_t(handler_down)] { BOOST_TEST(false); }, + [](std::exception_ptr) { BOOST_TEST(false); })( + parent(destroy_count_t(parent_down))); + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST_EQ(handler_down, 1); + } + + // The trampoline of run_async(ex, alloc) holds the allocator that + // frames below it deallocate through, so teardown from below must + // free each frame before destroying its parent. + void testTeardownFreesChildBeforeAllocatorOwner() + { + alloc_counts st; + int child_down = 0, parent_down = 0; + parked slot; + auto child = [](parked* s, destroy_count_t) -> task + { + co_await park_awaitable{s}; + }; + auto parent = [&](parked* s, destroy_count_t) + -> task + { + co_await child(s, destroy_count_t(child_down)); + }; + int n = 0; + test_executor ex(n); + run_async(ex, counting_allocator(st))( + parent(&slot, destroy_count_t(parent_down))); + BOOST_TEST(slot); + capy::abandon(slot); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // Every frame of a torn-down chain is freed before the frame that + // owns the chain's memory resource is destroyed. + void testTeardownFreesFramesBeforeResourceOwner() + { + lease_resource mr; + int leaf_down = 0, mid_down = 0, owner_down = 0; + parked slot; + std::queue q; + queuing_executor ex(q); + auto leaf = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto mid = [&](destroy_count_t) -> task + { + co_await leaf(&slot, destroy_count_t(leaf_down)); + }; + auto owner = [&](destroy_count_t) -> task + { + lease l(mr); + co_await capy::run(&mr)(mid(destroy_count_t(mid_down))); + }; + run_async(ex)(owner(destroy_count_t(owner_down))); + pump(q); + BOOST_TEST(slot); + + capy::abandon(slot); + pump(q); + BOOST_TEST_EQ(leaf_down, 1); + BOOST_TEST_EQ(mid_down, 1); + BOOST_TEST_EQ(owner_down, 1); + BOOST_TEST(mr.allocs.load() > 0); + BOOST_TEST(mr.balanced()); + } + + // Every frame of a torn-down chain is freed before run_async's + // handlers are destroyed with the launch that owns the resource; + // only that launch's own frame is freed after them. + void testTeardownFreesFramesBeforeLaunch() + { + lease_resource mr; + int leaf_down = 0, root_down = 0; + parked slot; + std::queue q; + queuing_executor ex(q); + auto leaf = [](parked* s, destroy_count_t) + -> task + { + co_await park_awaitable{s}; + }; + auto root = [&](destroy_count_t) -> task + { + co_await leaf(&slot, destroy_count_t(leaf_down)); + }; + run_async(ex, &mr, + [l = lease(mr)] { BOOST_TEST(false); }, + [](std::exception_ptr) { BOOST_TEST(false); })( + root(destroy_count_t(root_down))); + pump(q); + BOOST_TEST(slot); + + capy::abandon(slot); + pump(q); + BOOST_TEST_EQ(leaf_down, 1); + BOOST_TEST_EQ(root_down, 1); + BOOST_TEST(mr.allocs.load() > 1); + BOOST_TEST_EQ(mr.allocs.load(), mr.deallocs.load()); + BOOST_TEST_EQ(mr.late.load(), 1); + } + + // A chain comes down innermost first: each frame's locals, then its + // parameters, before the frame that awaited it, and the completion + // handlers last. + void testTeardownDestroysInnermostFirst() + { + std::vector log; + struct note + { + std::vector* log; + char const* what; + + note(std::vector& l, char const* w) noexcept + : log(&l) + , what(w) + { + } + + note(note&& o) noexcept + : log(std::exchange(o.log, nullptr)) + , what(o.what) + { + } + + ~note() + { + if(log) + log->push_back(what); + } + }; + parked slot; + { + auto leaf = [&](note) -> task + { + note local(log, "leaf local"); + co_await park_awaitable{&slot}; + }; + auto mid = [&](note) -> task + { + note local(log, "mid local"); + co_await leaf(note(log, "leaf param")); + }; + auto root = [&](note) -> task + { + note local(log, "root local"); + co_await mid(note(log, "mid param")); + }; + std::queue q; + queuing_executor ex(q); + run_async(ex, + [n = note(log, "handler")] { BOOST_TEST(false); }, + [](std::exception_ptr) { BOOST_TEST(false); })( + root(note(log, "root param"))); + pump(q); + BOOST_TEST(slot); + + capy::abandon(slot); + pump(q); + } + std::vector const expected{ + "leaf local", "leaf param", "mid local", "mid param", + "root local", "root param", "handler"}; + BOOST_TEST(log == expected); + } + + // A long chain is torn down once, every frame freed. + void testLongChainTeardown() + { + constexpr int depth = 100; + alloc_counts st; + int down = 0; + parked slot; + { + std::queue q; + queuing_executor ex(q); + run_async(ex, counting_allocator(st))( + nest_chain(depth, &slot, &down, destroy_count_t(down))); + pump(q); + BOOST_TEST(slot); + BOOST_TEST_EQ(down, 0); + + capy::abandon(slot); + pump(q); + } + BOOST_TEST_EQ(down, depth + 1); + BOOST_TEST(st.balanced()); + } + + // A by-value parameter that owns another frame is destroyed after + // the promise but before the frame is freed; freeing that frame must + // not destroy the parents of the frame still being torn down. + void testTeardownWithFrameOwningParameter() + { + alloc_counts st; + int sub_down = 0, parent_down = 0; + parked slot; + auto sub = [](destroy_count_t) -> task + { + co_return; + }; + auto leaf = [](task t, parked* s) + -> task + { + std::ignore = t; + co_await park_awaitable{s}; + }; + auto parent = [&](parked* s, destroy_count_t) + -> task + { + co_await leaf(sub(destroy_count_t(sub_down)), s); + }; + int n = 0; + test_executor ex(n); + run_async(ex, counting_allocator(st))( + parent(&slot, destroy_count_t(parent_down))); + BOOST_TEST(slot); + capy::abandon(slot); + BOOST_TEST_EQ(sub_down, 1); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // Through run(), the parent may belong to another context, so its + // destruction is posted to its own executor instead of done inline. + void testDestroyAwaitedChildThroughRun() + { + int child_down = 0, parent_down = 0; + std::queue q; + { + int dispatch_count = 0; + test_executor ex(dispatch_count); + queuing_executor caller(q); + auto child = [](destroy_count_t) -> capy::task + { + co_await self_destroy_awaitable{}; + }; + auto parent = [&](destroy_count_t) -> capy::task + { + co_await capy::run(ex)(child(destroy_count_t(child_down))); + }; + run_async(caller)(parent(destroy_count_t(parent_down))); + BOOST_TEST_EQ(q.size(), 1u); + auto start = q.front(); + q.pop(); + safe_resume(start.h); + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 0); + BOOST_TEST(!q.empty()); + pump(q); + } + BOOST_TEST_EQ(child_down, 1); + BOOST_TEST_EQ(parent_down, 1); + } + void testStop() { @@ -1280,6 +1645,12 @@ struct task_test void run() { + testMoveAssignDestroysHeldFrame(); +#if !defined(_WIN32) + testDestroyLentTaskTerminates(); + testMoveAssignOverLentTaskTerminates(); + testDestroyUnstartedOrFinishedTaskIsFine(); +#endif testBoolSuspendAwaitable(); testReturnValue(); testException(); @@ -1321,6 +1692,14 @@ struct task_test testTearDown(); testDelete(); + testDestroyAwaitedChild(); + testDestroyAwaitedChildThroughRun(); + testTeardownFreesChildBeforeAllocatorOwner(); + testTeardownFreesFramesBeforeResourceOwner(); + testTeardownFreesFramesBeforeLaunch(); + testTeardownDestroysInnermostFirst(); + testLongChainTeardown(); + testTeardownWithFrameOwningParameter(); testStop(); } }; diff --git a/test/unit/test/stream.cpp b/test/unit/test/stream.cpp index 4494b55b8..a80285237 100644 --- a/test/unit/test/stream.cpp +++ b/test/unit/test/stream.cpp @@ -28,11 +28,14 @@ #include "test/unit/test_helpers.hpp" #include +#include #include +#include #include #include #include #include +#include namespace boost { namespace capy { @@ -681,7 +684,7 @@ class stream_test // Driven manually on a queuing executor for deterministic, // non-blocking control (mirrors async_mutex's cancellation test). auto [a, b] = make_stream_pair(); - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; @@ -713,7 +716,7 @@ class stream_test ss.request_stop(); BOOST_TEST(! q.empty()); - q.front().resume(); + q.front().h.resume(); q.pop(); BOOST_TEST(reader_done); @@ -744,6 +747,47 @@ class stream_test BOOST_TEST(ran); } + void + testReadSomeStopRequestedResumesInline() + { + // With stop already requested the read never registers a stop + // callback: one would fire inside await_suspend and post the + // coroutine, which another thread could then resume and finish + // while await_suspend still writes to its frame. + auto [a, b] = make_stream_pair(); + std::queue q; + queuing_executor ex(q); + std::stop_source ss; + ss.request_stop(); + + std::error_code reader_ec; + bool reader_done = false; + + auto reader = [](stream& s, + std::error_code& out_ec, bool& done) -> task<> + { + char buf[32] = {}; + auto [ec, n] = co_await s.read_some(make_buffer(buf)); + out_ec = ec; + std::ignore = n; + done = true; + }(b, reader_ec, reader_done); + + auto h = reader.handle(); + reader.release(); + io_env env; + env.executor = executor_ref(ex); + env.stop_token = ss.get_token(); + h.promise().set_environment(&env); + + h.resume(); + BOOST_TEST(q.empty()); + BOOST_TEST(reader_done); + BOOST_TEST(reader_ec == make_error_code(error::canceled)); + + h.destroy(); + } + void testReadSomeDestroyWhileSuspended() { @@ -751,7 +795,7 @@ class stream_test // the stop callback and unlink the pending slot, so a later peer // operation does not touch a freed claim flag. auto [a, b] = make_stream_pair(); - std::queue> q; + std::queue q; queuing_executor ex(q); std::stop_source ss; @@ -866,6 +910,43 @@ class stream_test BOOST_TEST(ran); } + // A peer's write wakes a read parked on an executor that queues + // the continuation in place: the queued continuation is left as + // posted, and the reader resumes with the data. + void + testWakeOnInPlaceQueue() + { + thread_pool pool(1); + std::counting_semaphore<2> done(0); + auto [a, b] = make_stream_pair(); + std::size_t got = 0; + + auto reader = [&]() -> task<> + { + char buf[32] = {}; + auto [ec, n] = co_await b.read_some(make_buffer(buf)); + if(!ec) + got = n; + }; + auto writer = [&]() -> task<> + { + std::ignore = co_await a.write_some(make_buffer("hello", 5)); + }; + + // Posted in order to one thread: the reader parks first. + run_async(pool.get_executor(), + [&]() { done.release(); }, + [&](std::exception_ptr) { done.release(); })(reader()); + run_async(pool.get_executor(), + [&]() { done.release(); }, + [&](std::exception_ptr) { done.release(); })(writer()); + + // Only a watchdog: a reader that is never woken hangs. + for(int i = 0; i < 2; ++i) + BOOST_TEST(done.try_acquire_for(std::chrono::seconds(30))); + BOOST_TEST_EQ(got, 5u); + } + void run() { @@ -879,6 +960,7 @@ class stream_test testReadSomeEmptyNoData(); testMaxReadSize(); testMaxReadSizeMultiple(); + testWakeOnInPlaceQueue(); // Write operations testWriteSome(); @@ -901,6 +983,7 @@ class stream_test // Cancellation testReadSomeCancellationWhileSuspended(); testReadSomeStopRequestedBeforeSuspend(); + testReadSomeStopRequestedResumesInline(); testReadSomeDestroyWhileSuspended(); testReadSomeCancellationCrossThread(); testReadSomeStopWithDataAvailable(); diff --git a/test/unit/test_helpers.hpp b/test/unit/test_helpers.hpp index a0f102b6d..8be99caae 100644 --- a/test/unit/test_helpers.hpp +++ b/test/unit/test_helpers.hpp @@ -14,7 +14,9 @@ #include #include #include +#include #include +#include #include #include @@ -24,11 +26,18 @@ #include "test_suite.hpp" #include +#include #include +#include +#include +#include +#include #include #include #include #include +#include +#include // Valgrind slows execution ~10-20x; scale timing-sensitive // durations to avoid false failures. @@ -162,6 +171,13 @@ test_executor::context() const noexcept static_assert(Executor); static_assert(ExecutionContext); +/// A test context whose services the test shuts down directly. +class closable_test_context : public test_io_context +{ +public: + using execution_context::shutdown; +}; + //---------------------------------------------------------- // Wait Utilities @@ -242,14 +258,19 @@ thread_name_starts_with(char const* name_prefix) #endif // BOOST_CAPY_TEST_CAN_GET_THREAD_NAME +/** Gives up its own coroutine as a holder would. + + The awaiting coroutine is abandoned from inside `await_suspend`, so + its chain is unwound without ever resuming. +*/ struct self_destroy_awaitable { bool await_ready() {return false;} - std::coroutine_handle<> await_suspend(std::coroutine_handle<> h, io_env const*) + std::coroutine_handle<> await_suspend(std::coroutine_handle<> h, io_env const* env) { - // one wouldn't expect this to happen, but it should not cause UB - h.destroy(); + continuation c{h, nullptr, env->owner}; + capy::abandon(c); return std::noop_coroutine(); } void await_resume() {} @@ -278,7 +299,7 @@ post_resume( io_env const& env, std::coroutine_handle<> h) noexcept { - return resume_via_post{env.executor, continuation{h}}; + return resume_via_post{env.executor, continuation{h, nullptr, env.owner}}; } // test awaitable that must be stopped in order to resume. @@ -406,6 +427,78 @@ void_throws_exception(char const* msg) // Queuing Executor //---------------------------------------------------------- +/** A sized input range whose iterator holds the element it yields. + + Input iterators may do this: each increment replaces the element, + so a reference to one does not outlive the next step. +*/ +template +class stashing_range +{ + Make make_; + std::size_t n_; + +public: + class iterator + { + public: + using value_type = + std::invoke_result_t; + using difference_type = std::ptrdiff_t; + + private: + Make const* make_ = nullptr; + std::size_t i_ = 0; + std::size_t n_ = 0; + mutable std::optional stash_; + + void fill() + { + stash_.reset(); + if(i_ < n_) + stash_.emplace((*make_)(i_)); + } + + public: + iterator(Make const* make, std::size_t i, std::size_t n) + : make_(make) + , i_(i) + , n_(n) + { + fill(); + } + + iterator(iterator&&) = default; + iterator& operator=(iterator&&) = default; + + value_type& operator*() const { return *stash_; } + + iterator& operator++() + { + ++i_; + fill(); + return *this; + } + + void operator++(int) { ++*this; } + + bool operator==(std::default_sentinel_t) const noexcept + { + return i_ == n_; + } + }; + + stashing_range(Make make, std::size_t n) + : make_(std::move(make)) + , n_(n) + { + } + + iterator begin() const { return iterator(&make_, 0, n_); } + std::default_sentinel_t end() const noexcept { return {}; } + std::size_t size() const noexcept { return n_; } +}; + /** Queuing executor that allows controlled interleaving of tasks. Unlike test_executor which runs tasks synchronously, this executor @@ -414,10 +507,10 @@ void_throws_exception(char const* msg) */ struct queuing_executor { - std::queue>* queue_; + std::queue* queue_; test_io_context* ctx_ = nullptr; - explicit queuing_executor(std::queue>& q) + explicit queuing_executor(std::queue& q) : queue_(&q) { } @@ -437,18 +530,57 @@ struct queuing_executor std::coroutine_handle<> dispatch(continuation& c) const { - queue_->push(c.h); + queue_->push(c); return std::noop_coroutine(); } void post(continuation& c) const { - queue_->push(c.h); + queue_->push(c); } }; static_assert(Executor); +/** Executor that queues posts until closed, then abandons them. + + Models a context that has shut down: whatever is posted to it + afterwards is given up rather than run. +*/ +struct closing_executor +{ + std::queue* q; + bool* closed; + + bool operator==(closing_executor const& o) const noexcept + { + return q == o.q; + } + + test_io_context& context() const noexcept + { + return default_test_io_context(); + } + + void on_work_started() const noexcept {} + void on_work_finished() const noexcept {} + + std::coroutine_handle<> dispatch(continuation& c) const + { + return c.h; + } + + void post(continuation& c) const + { + if(*closed) + capy::abandon(c); + else + q->push(c); + } +}; + +static_assert(Executor); + //---------------------------------------------------------- // Yield Awaitable //---------------------------------------------------------- @@ -481,6 +613,218 @@ struct yield_awaitable } }; +/** Counts its own destruction; a frame parameter of this type + reports when the frame is destroyed. +*/ +struct destroy_count_t +{ + int* n; + explicit destroy_count_t(int& c) : n(&c) {} + destroy_count_t(destroy_count_t&& o) noexcept + : n(std::exchange(o.n, nullptr)) {} + ~destroy_count_t() + { + if(n) + ++*n; + } +}; + +/** A suspended coroutine as its holder keeps it. + + Its handle, to resume it, and its segment's owner, to + @ref abandon it. +*/ +struct parked : continuation +{ + /// Return true while a coroutine is parked here. + explicit operator bool() const noexcept + { + return static_cast(h); + } + + /// Resume the parked coroutine. + void resume() const + { + h.resume(); + } +}; + +/** Parks the awaiting coroutine, handing it to the test. + + The test then plays the holder: resume `*slot`, or abandon it as a + context's teardown would. +*/ +struct park_awaitable +{ + parked* slot; + + bool await_ready() const noexcept { return false; } + + std::coroutine_handle<> + await_suspend(std::coroutine_handle<> h, io_env const* env) noexcept + { + slot->h = h; + slot->owner = env->owner; + return std::noop_coroutine(); + } + + void await_resume() const noexcept {} +}; + +/// Counts of a @ref counting_allocator and its copies. +struct alloc_counts +{ + std::atomic allocs{0}; + std::atomic deallocs{0}; + std::atomic live{0}; + std::atomic late{0}; + + /// Return true once every allocation is freed, none late. + bool balanced() const noexcept + { + return allocs.load() == deallocs.load() && late.load() == 0; + } +}; + +/** An allocator whose copies count allocations and frees. + + A free made once every copy is destroyed went through an allocator + that no longer exists, as when a frame outlives the frame holding + its chain's allocator; it is counted as late. +*/ +template +struct counting_allocator +{ + using value_type = T; + + alloc_counts* c; + + ~counting_allocator() + { + --c->live; + } + + explicit counting_allocator(alloc_counts& counts) noexcept + : c(&counts) + { + ++c->live; + } + + counting_allocator(counting_allocator const& o) noexcept + : c(o.c) + { + ++c->live; + } + + template + counting_allocator(counting_allocator const& o) noexcept + : c(o.c) + { + ++c->live; + } + + counting_allocator& operator=(counting_allocator const&) = default; + + T* allocate(std::size_t n) + { + ++c->allocs; + return static_cast(::operator new(n * sizeof(T))); + } + + void deallocate(T* p, std::size_t) noexcept + { + ++c->deallocs; + if(c->live.load() == 0) + ++c->late; + ::operator delete(p); + } + + template + bool operator==(counting_allocator const& o) const noexcept + { + return c == o.c; + } +}; + +/** A memory resource that counts frees made after its lease ended. + + A frame that owns a chain's allocator must outlive every frame + allocated from it. A @ref lease held in that frame ends when the + frame's locals are destroyed; a free counted as late came after. +*/ +struct lease_resource : std::pmr::memory_resource +{ + std::atomic leased{true}; + std::atomic allocs{0}; + std::atomic deallocs{0}; + std::atomic late{0}; + + /// Return true once every allocation is freed, none late. + bool balanced() const noexcept + { + return allocs.load() == deallocs.load() && late.load() == 0; + } + +protected: + // Forwarded through another resource, which the optimizer cannot see + // into, so it does not pair the frame's operator delete with the + // global operator new. + void* do_allocate(std::size_t n, std::size_t a) override + { + ++allocs; + return std::pmr::new_delete_resource()->allocate(n, a); + } + + void do_deallocate(void* p, std::size_t n, std::size_t a) override + { + ++deallocs; + if(!leased.load()) + ++late; + std::pmr::new_delete_resource()->deallocate(p, n, a); + } + + bool do_is_equal(memory_resource const& o) const noexcept override + { + return this == &o; + } +}; + +/// Ends the lease on a @ref lease_resource when destroyed. +struct lease +{ + lease_resource* r; + + explicit lease(lease_resource& res) noexcept + : r(&res) + { + } + + lease(lease&& o) noexcept + : r(std::exchange(o.r, nullptr)) + { + } + + lease& operator=(lease&&) = delete; + + ~lease() + { + if(r) + r->leased.store(false); + } +}; + +/// Resume queued continuations in FIFO order until the queue is empty. +inline void +pump(std::queue& q) +{ + while(!q.empty()) + { + auto c = q.front(); + q.pop(); + safe_resume(c.h); + } +} + } // capy } // boost diff --git a/test/unit/when_all.cpp b/test/unit/when_all.cpp index 79d6d5bb2..b99ff3fc7 100644 --- a/test/unit/when_all.cpp +++ b/test/unit/when_all.cpp @@ -12,7 +12,11 @@ #include #include +#include +#include +#include #include +#include #include #include #include @@ -22,6 +26,9 @@ #include #include +#include +#include +#include #include #include #include @@ -252,6 +259,98 @@ struct when_all_range_test BOOST_TEST(completed); } + // A range whose iterator holds the element it yields: each + // element is taken before the next step replaces it. + void + testStashingRange() + { + int dc = 0; + test_executor ex(dc); + bool completed = false; + + stashing_range tasks([](std::size_t i) + { + return io_success_size((i + 1) * 10); + }, 3); + static_assert(!std::ranges::forward_range); + + run_async(ex, + [&](io_result> r) { + completed = true; + BOOST_TEST(!std::get<0>(r)); + BOOST_TEST_EQ(std::get<1>(r).size(), 3u); + BOOST_TEST_EQ(std::get<1>(r)[0], 10u); + BOOST_TEST_EQ(std::get<1>(r)[1], 20u); + BOOST_TEST_EQ(std::get<1>(r)[2], 30u); + }, + [](std::exception_ptr) {})( + when_all(tasks)); + + BOOST_TEST(completed); + } + + // A view yielding its awaitables by value: each lives only for its + // step of the iteration. + // A range that throws while when_all takes it destroys the + // awaitables already taken and reports the exception. + void + testRangeThrowsWhileTaken() + { + int dc = 0; + test_executor ex(dc); + int down = 0; + bool completed = false; + bool threw = false; + auto body = [](destroy_count_t) -> io_task + { + co_return io_result{std::error_code(), 1}; + }; + auto tasks = std::views::iota(0, 3) | + std::views::transform([&](int i) + { + if(i == 1) + throw test_exception("range"); + return body(destroy_count_t(down)); + }); + + run_async(ex, + [&](io_result>) { completed = true; }, + [&](std::exception_ptr) { threw = true; })( + when_all(tasks)); + + BOOST_TEST(!completed); + BOOST_TEST(threw); + BOOST_TEST_EQ(down, 1); + } + + void + testPrvalueRange() + { + int dc = 0; + test_executor ex(dc); + bool completed = false; + + auto tasks = std::views::iota(1, 4) | + std::views::transform([](int i) + { + return io_success_size(static_cast(i * 10)); + }); + + run_async(ex, + [&](io_result> r) { + completed = true; + BOOST_TEST(!std::get<0>(r)); + BOOST_TEST_EQ(std::get<1>(r).size(), 3u); + BOOST_TEST_EQ(std::get<1>(r)[0], 10u); + BOOST_TEST_EQ(std::get<1>(r)[1], 20u); + BOOST_TEST_EQ(std::get<1>(r)[2], 30u); + }, + [](std::exception_ptr) {})( + when_all(tasks)); + + BOOST_TEST(completed); + } + void testEmptyRange() { @@ -669,6 +768,9 @@ struct when_all_range_test testStopAlreadyRequestedTuple(); testSingleElement(); testMultipleElements(); + testPrvalueRange(); + testRangeThrowsWhileTaken(); + testStashingRange(); testEmptyRange(); testVoidRange(); testEmptyVoidRange(); @@ -1199,6 +1301,419 @@ struct when_all_io_result_test BOOST_TEST(result_ec == cond::eof); } + // A runner its context destroys before it first runs is a lost + // branch; the parent goes with the last one. + void testRunnersDestroyedBeforeStartDestroyParent() + { + int a_down = 0, b_down = 0, parent_down = 0; + std::queue q; + auto branch = [](destroy_count_t) -> task> + { + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_all( + branch(destroy_count_t(a_down)), + branch(destroy_count_t(b_down))); + }; + run_async(queuing_executor(q))(parent(destroy_count_t(parent_down))); + BOOST_TEST_EQ(q.size(), 1u); + auto start = q.front(); + q.pop(); + safe_resume(start.h); // parent posts both runners + BOOST_TEST_EQ(q.size(), 2u); + + // An unstarted runner owns no branch: the branches stay with the + // join until it is unwound. + capy::abandon(q.front()); + q.pop(); + BOOST_TEST_EQ(a_down, 0); + BOOST_TEST_EQ(b_down, 0); + BOOST_TEST_EQ(parent_down, 0); + + capy::abandon(q.front()); + q.pop(); + pump(q); + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(b_down, 1); + BOOST_TEST_EQ(parent_down, 1); + } + + // Both branches torn down from below: the combinator and the + // parent go with the last one. + void testTeardownOfAllBranchesDestroysParent() + { + int a_down = 0, b_down = 0, parent_down = 0; + parked sa, sb; + auto branch = [](parked* s, destroy_count_t) + -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_all( + branch(&sa, destroy_count_t(a_down)), + branch(&sb, destroy_count_t(b_down))); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + BOOST_TEST(sa && sb); + capy::abandon(sa); + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(parent_down, 0); // b still holds a share + capy::abandon(sb); + BOOST_TEST_EQ(b_down, 1); + BOOST_TEST_EQ(parent_down, 1); + } + + // A branch that completes after a sibling was lost destroys the + // parent instead of resuming it. + void testLostBranchParentIsDestroyedNotResumed() + { + int parent_down = 0; + bool resumed = false; + parked sa, sb; + auto branch = [](parked* s) -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_all(branch(&sa), branch(&sb)); + resumed = true; + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + capy::abandon(sa); + sb.resume(); + BOOST_TEST(!resumed); + BOOST_TEST_EQ(parent_down, 1); + } + + // Parks the when_all runner itself, so tearing it down enters + // through the runner's promise rather than through a branch task. + struct park_io_awaitable : park_awaitable + { + io_result<> await_resume() const noexcept { return {}; } + }; + + // Losing every branch under run_async(ex, alloc) frees the runner + // frames before the trampoline holding their allocator. + void testLostBranchesFreeBeforeAllocatorOwner() + { + alloc_counts st; + int parent_down = 0; + parked sa, sb; + auto parent = [](parked* a, + parked* b, destroy_count_t) -> task + { + std::ignore = co_await when_all( + park_io_awaitable{{a}}, park_io_awaitable{{b}}); + }; + int n = 0; + test_executor ex(n); + run_async(ex, counting_allocator(st))( + parent(&sa, &sb, destroy_count_t(parent_down))); + capy::abandon(sa); + capy::abandon(sb); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // Branch tasks lost under run(mr) are freed before the frame that + // owns the resource is destroyed. + void testLostTaskBranchesFreedBeforeResourceOwner() + { + lease_resource mr; + int a_down = 0, b_down = 0, owner_down = 0, n = 0; + parked sa, sb; + test_executor ex(n); + auto branch = [](parked* s, destroy_count_t) + -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto join = [&]() -> task + { + std::ignore = co_await when_all( + branch(&sa, destroy_count_t(a_down)), + branch(&sb, destroy_count_t(b_down))); + }; + auto owner = [&](destroy_count_t) -> task + { + lease l(mr); + co_await capy::run(&mr)(join()); + }; + run_async(ex)(owner(destroy_count_t(owner_down))); + BOOST_TEST(sa && sb); + capy::abandon(sa); + capy::abandon(sb); + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(b_down, 1); + BOOST_TEST_EQ(owner_down, 1); + BOOST_TEST(mr.allocs.load() > 0); + BOOST_TEST(mr.balanced()); + } + + // The range overload tears down like the variadic one: no branch + // destroys a sibling, and the parent goes with the last branch. + void testRangeTeardownOfAllBranchesDestroysParent() + { + constexpr int count = 3; + int down[count] = {}, parent_down = 0; + parked slots[count]; + auto branch = [](parked* s, destroy_count_t) + -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::vector>> tasks; + for(int i = 0; i < count; ++i) + tasks.push_back( + branch(&slots[i], destroy_count_t(down[i]))); + std::ignore = co_await when_all(std::move(tasks)); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + for(int i = 0; i < count; ++i) + { + BOOST_TEST(slots[i]); + capy::abandon(slots[i]); + for(int j = 0; j < count; ++j) + BOOST_TEST_EQ(down[j], j <= i ? 1 : 0); + BOOST_TEST_EQ(parent_down, i == count - 1 ? 1 : 0); + } + } + + // A branch that finished before its sibling was lost does not keep + // the parent alive or resume it. + void testFinishedThenLostDestroysParent() + { + int parent_down = 0; + bool resumed = false; + parked sa, sb; + auto branch = [](parked* s) -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_all(branch(&sa), branch(&sb)); + resumed = true; + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + sa.resume(); + BOOST_TEST_EQ(parent_down, 0); + capy::abandon(sb); + BOOST_TEST(!resumed); + BOOST_TEST_EQ(parent_down, 1); + } + + // A join inside a branch of another: the outer parent goes only + // once every leaf of both joins is torn down. + void testNestedJoinsTeardown() + { + int leaf_down = 0, inner_down = 0, parent_down = 0; + parked sa, sb, sc; + auto branch = [](parked* s, destroy_count_t) -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto inner = [&](destroy_count_t) -> task> + { + std::ignore = co_await when_all( + branch(&sb, destroy_count_t(leaf_down)), + branch(&sc, destroy_count_t(leaf_down))); + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_all( + branch(&sa, destroy_count_t(leaf_down)), + inner(destroy_count_t(inner_down))); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + BOOST_TEST(sa && sb && sc); + capy::abandon(sb); + capy::abandon(sa); + BOOST_TEST_EQ(leaf_down, 2); + BOOST_TEST_EQ(inner_down, 0); + BOOST_TEST_EQ(parent_down, 0); + capy::abandon(sc); + BOOST_TEST_EQ(leaf_down, 3); + BOOST_TEST_EQ(inner_down, 1); + BOOST_TEST_EQ(parent_down, 1); + } + + // A stop that loses the last branch while the join is still + // stopping its branches leaves the join's stop state alive until + // that request returns. Freeing it early shows only under a + // sanitizer, and only libstdc++'s callback order makes it reachable. + void testStopLosingLastBranch() + { + std::queue q; + bool closed = false; + int parent_down = 0; + async_event ev; + std::stop_source src; + auto branch = [&]() -> task> + { + auto tok = (co_await this_coro::environment)->stop_token; + // Registered before the wait's, so libstdc++ runs it + // after the wait's and relocks the stop state between. + std::stop_callback extra(tok, [] {}); + auto [ec] = co_await ev.wait(); + co_return io_result<>{ec}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_all(branch(), branch()); + }; + run_async(closing_executor{&q, &closed}, src.get_token())( + parent(destroy_count_t(parent_down))); + pump(q); + BOOST_TEST_EQ(parent_down, 0); + closed = true; + src.request_stop(); + BOOST_TEST_EQ(parent_down, 1); + } + + // A branch that hops executors through run is torn down below the + // hop: the branch is lost once it is back on the caller's executor. + void testRunInsideBranchTornDown() + { + int leaf_down = 0, hop_down = 0, parent_down = 0, n = 0; + parked sa, sb; + std::queue q; + queuing_executor caller(q); + test_executor inner(n); + auto leaf = [](parked* s, destroy_count_t) -> task + { + co_await park_awaitable{s}; + }; + auto hop = [&](parked* s, destroy_count_t) -> task> + { + co_await capy::run(inner)(leaf(s, destroy_count_t(leaf_down))); + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_all( + hop(&sa, destroy_count_t(hop_down)), + hop(&sb, destroy_count_t(hop_down))); + }; + run_async(caller)(parent(destroy_count_t(parent_down))); + pump(q); + BOOST_TEST(sa && sb); + + capy::abandon(sa); + BOOST_TEST_EQ(leaf_down, 1); + BOOST_TEST_EQ(hop_down, 0); // lost on the caller's executor + BOOST_TEST(!q.empty()); + pump(q); + BOOST_TEST_EQ(hop_down, 1); + BOOST_TEST_EQ(parent_down, 0); + + capy::abandon(sb); + pump(q); + BOOST_TEST_EQ(leaf_down, 2); + BOOST_TEST_EQ(hop_down, 2); + BOOST_TEST_EQ(parent_down, 1); + } + + // Runners abandoned before they start free their branches before + // the last loss unwinds the frame that owns the branches' memory. + void testUnstartedRunnersFreeBeforeResourceOwner() + { + for(bool range : {false, true}) + { + lease_resource mr; + std::queue q; + bool closed = false; + int owner_down = 0; + auto branch = []() -> task> + { + co_return io_result<>{}; + }; + auto join = [&]() -> task + { + if(range) + { + std::vector>> tasks; + tasks.push_back(branch()); + tasks.push_back(branch()); + std::ignore = co_await when_all(std::move(tasks)); + } + else + { + std::ignore = co_await when_all(branch(), branch()); + } + }; + auto owner = [&](destroy_count_t) -> task + { + lease l(mr); + co_await capy::run(&mr)(join()); + }; + run_async(closing_executor{&q, &closed})( + owner(destroy_count_t(owner_down))); + BOOST_TEST_EQ(q.size(), 2u); // both runners posted + closed = true; + while(!q.empty()) + { + auto c = q.front(); + q.pop(); + capy::abandon(c); + } + BOOST_TEST_EQ(owner_down, 1); + BOOST_TEST(mr.balanced()); + } + } + + // Losing a branch asks the others to stop. + void testLostBranchRequestsStop() + { + parked sa, sb; + std::stop_token seen; + auto branch = [](parked* s, std::stop_token* out) + -> task> + { + if(out) + *out = (co_await this_coro::environment)->stop_token; + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&]() -> task + { + std::ignore = co_await when_all( + branch(&sa, nullptr), branch(&sb, &seen)); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent()); + BOOST_TEST(!seen.stop_requested()); + capy::abandon(sa); + BOOST_TEST(seen.stop_requested()); + capy::abandon(sb); + } + void run() { @@ -1222,6 +1737,18 @@ struct when_all_io_result_test testMultiValueChild(); testVoidResults(); testFirstErrorInTimeWins(); + testRunnersDestroyedBeforeStartDestroyParent(); + testTeardownOfAllBranchesDestroysParent(); + testLostBranchParentIsDestroyedNotResumed(); + testLostBranchRequestsStop(); + testUnstartedRunnersFreeBeforeResourceOwner(); + testRunInsideBranchTornDown(); + testStopLosingLastBranch(); + testLostTaskBranchesFreedBeforeResourceOwner(); + testRangeTeardownOfAllBranchesDestroysParent(); + testFinishedThenLostDestroysParent(); + testNestedJoinsTeardown(); + testLostBranchesFreeBeforeAllocatorOwner(); } }; diff --git a/test/unit/when_any.cpp b/test/unit/when_any.cpp index 55940177f..c207a3bf2 100644 --- a/test/unit/when_any.cpp +++ b/test/unit/when_any.cpp @@ -13,6 +13,9 @@ #include #include +#include +#include +#include #include #include #include @@ -21,6 +24,7 @@ #include "test_helpers.hpp" #include "test_suite.hpp" +#include #include #include #include @@ -28,6 +32,7 @@ #include #include #include +#include namespace boost { namespace capy { @@ -103,17 +108,18 @@ struct immediate_io_awaitable int g_throwing_move_count = 0; -// A payload whose move constructor throws on the second and later moves. -// The io_result constructor performs the first move; the extract and -// emplace into when_any's result variant are the second/third moves, -// inside when_any's try/catch — exercising the winner-exception path. +// A payload whose move constructor throws on the third and later moves. +// The io_result constructor performs the first move and the branch +// taking the awaited result the second; the extract into when_any's +// result variant is the third, inside when_any's try/catch, exercising +// the winner-exception path. struct throwing_move_payload { int v; explicit throwing_move_payload(int x) noexcept : v(x) {} throwing_move_payload(throwing_move_payload&& o) : v(o.v) { - if(++g_throwing_move_count >= 2) + if(++g_throwing_move_count >= 3) throw std::runtime_error("payload move"); } throwing_move_payload& @@ -217,6 +223,62 @@ struct when_any_vector_test BOOST_TEST(completed); } + // A range whose iterator holds the element it yields: each + // element is taken before the next step replaces it. + void + testStashingRange() + { + int dc = 0; + test_executor ex(dc); + bool completed = false; + + stashing_range tasks([](std::size_t i) + { + return io_success_size((i + 1) * 10); + }, 3); + static_assert(!std::ranges::forward_range); + + run_async(ex, + [&](std::variant> v) { + completed = true; + BOOST_TEST_EQ(v.index(), 1u); + auto [idx, val] = std::get<1>(v); + BOOST_TEST_EQ(val, (idx + 1) * 10); + }, + [](std::exception_ptr) {})( + when_any(tasks)); + + BOOST_TEST(completed); + } + + // A view yielding its awaitables by value: each lives only for its + // step of the iteration. + void + testPrvalueRange() + { + int dc = 0; + test_executor ex(dc); + bool completed = false; + + auto tasks = std::views::iota(1, 4) | + std::views::transform([](int i) + { + return io_success_size(static_cast(i * 10)); + }); + + run_async(ex, + [&](std::variant> v) { + completed = true; + BOOST_TEST_EQ(v.index(), 1u); + auto [idx, val] = std::get<1>(v); + BOOST_TEST_EQ(val, (idx + 1) * 10); + }, + [](std::exception_ptr) {})( + when_any(tasks)); + + BOOST_TEST(completed); + } + void testEmptyVectorThrows() { @@ -478,7 +540,7 @@ struct when_any_vector_test void testLongLivedTasksCancelled() { - std::queue> work_queue; + std::queue work_queue; queuing_executor ex(work_queue); std::atomic cancelled_count{0}; @@ -519,11 +581,7 @@ struct when_any_vector_test [](std::exception_ptr) {})( when_any(std::move(tasks))); - while (!work_queue.empty()) { - auto h = work_queue.front(); - work_queue.pop(); - h.resume(); - } + pump(work_queue); BOOST_TEST(when_any_completed); BOOST_TEST_EQ(completed_normally_count.load(), 1); @@ -595,6 +653,8 @@ struct when_any_vector_test { testSingleTaskSuccess(); testMultipleTasksFirstSuccessWins(); + testPrvalueRange(); + testStashingRange(); testEmptyVectorThrows(); testVoidTasksSuccess(); testVoidTasksAllThrow(); @@ -1018,9 +1078,387 @@ struct when_any_io_result_test BOOST_TEST(threw); } + // Runners destroyed before they start count as lost branches: the + // parent goes with the last of them. + void testRunnersDestroyedBeforeStartDestroyParent() + { + int a_down = 0, b_down = 0, parent_down = 0; + std::queue q; + auto branch = [](destroy_count_t) -> task> + { + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_any( + branch(destroy_count_t(a_down)), + branch(destroy_count_t(b_down))); + }; + run_async(queuing_executor(q))(parent(destroy_count_t(parent_down))); + BOOST_TEST_EQ(q.size(), 1u); + auto start = q.front(); + q.pop(); + safe_resume(start.h); // parent posts both runners + BOOST_TEST_EQ(q.size(), 2u); + auto ra = q.front(); + q.pop(); + auto rb = q.front(); + q.pop(); + + // An unstarted runner owns no branch: the branches stay with the + // join until it is unwound. + capy::abandon(ra); + pump(q); + BOOST_TEST_EQ(a_down, 0); + BOOST_TEST_EQ(b_down, 0); + BOOST_TEST_EQ(parent_down, 0); + + capy::abandon(rb); + pump(q); + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(b_down, 1); + BOOST_TEST_EQ(parent_down, 1); + } + + // Branch tasks lost under run(mr) are freed before the frame that + // owns the resource is destroyed. + void testLostTaskBranchesFreedBeforeResourceOwner() + { + lease_resource mr; + int a_down = 0, b_down = 0, owner_down = 0, n = 0; + parked sa, sb; + test_executor ex(n); + auto branch = [](parked* s, destroy_count_t) + -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto join = [&]() -> task + { + std::ignore = co_await when_any( + branch(&sa, destroy_count_t(a_down)), + branch(&sb, destroy_count_t(b_down))); + }; + auto owner = [&](destroy_count_t) -> task + { + lease l(mr); + co_await capy::run(&mr)(join()); + }; + run_async(ex)(owner(destroy_count_t(owner_down))); + BOOST_TEST(sa && sb); + capy::abandon(sa); + capy::abandon(sb); + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(b_down, 1); + BOOST_TEST_EQ(owner_down, 1); + BOOST_TEST(mr.allocs.load() > 0); + BOOST_TEST(mr.balanced()); + } + + // The range overload tears down like the variadic one: no branch + // destroys a sibling, and the parent goes with the last branch. + void testRangeTeardownOfAllBranchesDestroysParent() + { + constexpr int count = 3; + int down[count] = {}, parent_down = 0; + parked slots[count]; + auto branch = [](parked* s, destroy_count_t) + -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::vector>> tasks; + for(int i = 0; i < count; ++i) + tasks.push_back( + branch(&slots[i], destroy_count_t(down[i]))); + std::ignore = co_await when_any(std::move(tasks)); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + for(int i = 0; i < count; ++i) + { + BOOST_TEST(slots[i]); + capy::abandon(slots[i]); + for(int j = 0; j < count; ++j) + BOOST_TEST_EQ(down[j], j <= i ? 1 : 0); + BOOST_TEST_EQ(parent_down, i == count - 1 ? 1 : 0); + } + } + + // A winner does not resume the parent while a sibling is still out; + // losing that sibling destroys the parent instead. + void testWinnerThenLostDestroysParent() + { + int parent_down = 0; + bool resumed = false; + parked sa, sb; + auto branch = [](parked* s) -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_any(branch(&sa), branch(&sb)); + resumed = true; + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + sa.resume(); + BOOST_TEST(!resumed); + BOOST_TEST_EQ(parent_down, 0); + capy::abandon(sb); + BOOST_TEST(!resumed); + BOOST_TEST_EQ(parent_down, 1); + } + + // Both branches torn down from below: the combinator and the + // parent go with the last one. + void testTeardownOfAllBranchesDestroysParent() + { + int a_down = 0, b_down = 0, parent_down = 0; + parked sa, sb; + auto branch = [](parked* s, destroy_count_t) + -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_any( + branch(&sa, destroy_count_t(a_down)), + branch(&sb, destroy_count_t(b_down))); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + BOOST_TEST(sa && sb); + capy::abandon(sa); + BOOST_TEST_EQ(a_down, 1); + BOOST_TEST_EQ(parent_down, 0); // b still holds a share + capy::abandon(sb); + BOOST_TEST_EQ(b_down, 1); + BOOST_TEST_EQ(parent_down, 1); + } + + // A branch that finishes after a sibling was lost destroys the + // parent instead of resuming it. + void testLostBranchParentIsDestroyedNotResumed() + { + int parent_down = 0; + bool resumed = false; + parked sa, sb; + auto park = [](parked* s) -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto failing = [](parked* s) -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{make_error_code(error::canceled)}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_any(park(&sa), failing(&sb)); + resumed = true; + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + capy::abandon(sa); + sb.resume(); + BOOST_TEST(!resumed); + BOOST_TEST_EQ(parent_down, 1); + } + + // A branch that would win after its sibling was lost does not + // resume the parent: the parent goes with the join instead. + void testSuccessAfterLossDestroysParent() + { + int parent_down = 0; + bool resumed = false; + parked sa, sb; + auto branch = [](parked* s) -> task> + { + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_any(branch(&sa), branch(&sb)); + resumed = true; + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent(destroy_count_t(parent_down))); + capy::abandon(sa); + BOOST_TEST_EQ(parent_down, 0); + sb.resume(); + BOOST_TEST(!resumed); + BOOST_TEST_EQ(parent_down, 1); + } + + // Parks the when_any runner itself, so tearing it down enters + // through the runner's promise rather than through a branch task. + struct park_io_awaitable : park_awaitable + { + io_result<> await_resume() const noexcept { return {}; } + }; + + // Losing every branch under run_async(ex, alloc) frees the runner + // frames before the trampoline holding their allocator. + void testLostBranchesFreeBeforeAllocatorOwner() + { + alloc_counts st; + int parent_down = 0; + parked sa, sb; + auto parent = [](parked* a, + parked* b, destroy_count_t) -> task + { + std::ignore = co_await when_any( + park_io_awaitable{{a}}, park_io_awaitable{{b}}); + }; + int n = 0; + test_executor ex(n); + run_async(ex, counting_allocator(st))( + parent(&sa, &sb, destroy_count_t(parent_down))); + capy::abandon(sa); + capy::abandon(sb); + BOOST_TEST_EQ(parent_down, 1); + BOOST_TEST(st.allocs > 0); + BOOST_TEST(st.balanced()); + } + + // A stop that loses the last branch while the join is still + // stopping its branches leaves the join's stop state alive until + // that request returns. Freeing it early shows only under a + // sanitizer, and only libstdc++'s callback order makes it reachable. + void testStopLosingLastBranch() + { + std::queue q; + bool closed = false; + int parent_down = 0; + async_event ev; + std::stop_source src; + auto branch = [&]() -> task> + { + auto tok = (co_await this_coro::environment)->stop_token; + // Registered before the wait's, so libstdc++ runs it + // after the wait's and relocks the stop state between. + std::stop_callback extra(tok, [] {}); + auto [ec] = co_await ev.wait(); + co_return io_result<>{ec}; + }; + auto parent = [&](destroy_count_t) -> task + { + std::ignore = co_await when_any(branch(), branch()); + }; + run_async(closing_executor{&q, &closed}, src.get_token())( + parent(destroy_count_t(parent_down))); + pump(q); + BOOST_TEST_EQ(parent_down, 0); + closed = true; + src.request_stop(); + BOOST_TEST_EQ(parent_down, 1); + } + + // Runners abandoned before they start free their branches before + // the last loss unwinds the frame that owns the branches' memory. + void testUnstartedRunnersFreeBeforeResourceOwner() + { + for(bool range : {false, true}) + { + lease_resource mr; + std::queue q; + bool closed = false; + int owner_down = 0; + auto branch = []() -> task> + { + co_return io_result<>{}; + }; + auto join = [&]() -> task + { + if(range) + { + std::vector>> tasks; + tasks.push_back(branch()); + tasks.push_back(branch()); + std::ignore = co_await when_any(std::move(tasks)); + } + else + { + std::ignore = co_await when_any(branch(), branch()); + } + }; + auto owner = [&](destroy_count_t) -> task + { + lease l(mr); + co_await capy::run(&mr)(join()); + }; + run_async(closing_executor{&q, &closed})( + owner(destroy_count_t(owner_down))); + BOOST_TEST_EQ(q.size(), 2u); // both runners posted + closed = true; + while(!q.empty()) + { + auto c = q.front(); + q.pop(); + capy::abandon(c); + } + BOOST_TEST_EQ(owner_down, 1); + BOOST_TEST(mr.balanced()); + } + } + + // Losing a branch asks the others to stop. + void testLostBranchRequestsStop() + { + parked sa, sb; + std::stop_token seen; + auto branch = [](parked* s, std::stop_token* out) + -> task> + { + if(out) + *out = (co_await this_coro::environment)->stop_token; + co_await park_awaitable{s}; + co_return io_result<>{}; + }; + auto parent = [&]() -> task + { + std::ignore = co_await when_any( + branch(&sa, nullptr), branch(&sb, &seen)); + }; + int n = 0; + test_executor ex(n); + run_async(ex)(parent()); + BOOST_TEST(!seen.stop_requested()); + capy::abandon(sa); + BOOST_TEST(seen.stop_requested()); + capy::abandon(sb); + } + void run() { + testTeardownOfAllBranchesDestroysParent(); + testLostBranchParentIsDestroyedNotResumed(); + testSuccessAfterLossDestroysParent(); + testLostBranchRequestsStop(); + testUnstartedRunnersFreeBeforeResourceOwner(); + testStopLosingLastBranch(); + testRunnersDestroyedBeforeStartDestroyParent(); + testLostTaskBranchesFreedBeforeResourceOwner(); + testRangeTeardownOfAllBranchesDestroysParent(); + testWinnerThenLostDestroysParent(); + testLostBranchesFreeBeforeAllocatorOwner(); testWinnerPayloadMoveThrows(); testStopAlreadyRequested(); testFirstSuccessWins();