From 189986ec05ac0b91f46c673d4258e33bf6371afc Mon Sep 17 00:00:00 2001 From: Michael Vandeberg Date: Sat, 17 Oct 2026 12:52:58 -0600 Subject: [PATCH] docs: describe thread_pool shutdown accurately Rewrite the ~thread_pool documentation. It gives the steps in order: stop the pool, join the worker threads, then shut down the services and abandon the queued work. The destructor does not wait for outstanding work. Each abandoned chain unwinds on the executor of the coroutine that started it. The documentation also gives the relation to join(). Add a precondition: a thread of the pool must not destroy the pool. Correct these other statements in thread_pool.hpp: - The destructor precondition names only run_async. The run function does not add outstanding work to the pool. The precondition also tells the user not to call stop(), because join() then does not wait for outstanding work. - join() has no internal work guard to release. - A join() call that occurs while another join() call blocks returns immediately. Only the first call joins the threads. - post() does not run a continuation after the pool stops. The destructor abandons it. dispatch() does the same when it posts. - post() and dispatch() can throw std::system_error when they start the worker threads. Then the continuation is not queued. - The executor_type brief gives what the class does. Correct two example comments and the allocators page. The destructor abandons queued work. It does not discard it. pool.join() does not release the task frames. It returns after the task releases them. Closes #404 --- .../pages/4.coroutines/4h.allocators.adoc | 2 +- include/boost/capy/ex/thread_pool.hpp | 85 ++++++++++++++----- test/doc/programs/quick_start_hello.cpp | 3 +- test/doc/reference/thread_pool.record.cpp | 2 +- 4 files changed, 68 insertions(+), 24 deletions(-) diff --git a/doc/modules/ROOT/pages/4.coroutines/4h.allocators.adoc b/doc/modules/ROOT/pages/4.coroutines/4h.allocators.adoc index 59e70e772..210139ead 100644 --- a/doc/modules/ROOT/pages/4.coroutines/4h.allocators.adoc +++ b/doc/modules/ROOT/pages/4.coroutines/4h.allocators.adoc @@ -63,7 +63,7 @@ Run the same task eight times through it, one run at a time: include::example$snippets/4h_allocators.cpp[tag=recycling_observe,indent=0] ---- -The first run finds an empty pool and takes its coroutine frames from the heap. Each `pool.join()` returns those frames to the resource, so the next cpp:run_async[] call reuses them. Once the pool is warm, the upstream count stops climbing while the task keeps running -- the seven warm runs allocate nothing new. cpp:recycling_memory_resource[] does the same thing with size-class freelists and a lock-free thread-local cache. +The first run finds an empty pool and takes its coroutine frames from the heap. Each `pool.join()` returns after the task has finished and its frames are back in the resource, so the next cpp:run_async[] call reuses them. Once the pool is warm, the upstream count stops climbing while the task keeps running -- the seven warm runs allocate nothing new. cpp:recycling_memory_resource[] does the same thing with size-class freelists and a lock-free thread-local cache. == HALO Optimization diff --git a/include/boost/capy/ex/thread_pool.hpp b/include/boost/capy/ex/thread_pool.hpp index 4dc84b317..f317cf875 100644 --- a/include/boost/capy/ex/thread_pool.hpp +++ b/include/boost/capy/ex/thread_pool.hpp @@ -54,16 +54,37 @@ class BOOST_CAPY_DECL /** Destroy the thread pool. - Signals all worker threads to stop, waits for them to - finish, and destroys any pending work items. + Stops the pool, joins its worker threads, then abandons any + work that has not started: + + @li Calls @ref stop. Each worker thread finishes the item it + is running and exits; queued work is not run. + @li Joins the worker threads. Unlike @ref join, this does not + wait for outstanding work. If @ref join was already called, + the threads are already joined and this step does nothing. + @li Shuts down the pool's services, abandons every coroutine + still queued on the pool or its strands, then destroys the + services. + Each abandoned chain is unwound, without delivering a + result, on the executor of the coroutine that started it. + That happens during this destructor when the executor is + this pool, and later when it belongs to another execution + context. + See @ref chain_owner. + + To let queued work finish first, call @ref join before the + pool is destroyed. @pre No thread outside this pool may post or dispatch work to it (or to a strand built on it) concurrently with, or after, destruction. Doing so is undefined behavior. Submit such work - through @ref run_async or @ref run and call @ref join before - the pool is destroyed, so it has completed first. A thread - owned by one of this pool's services may post or dispatch - until that service's `shutdown()` returns. + through @ref run_async and call @ref join, without calling + @ref stop, before the pool is destroyed, so it has completed + first. A thread owned by one of this pool's services may post + or dispatch until that service's `shutdown()` returns. + + @pre Must not be called from a thread in this pool. A worker + thread cannot join itself, so this calls `std::terminate`. */ ~thread_pool(); @@ -103,8 +124,8 @@ class BOOST_CAPY_DECL /** Wait for all outstanding work to complete. - Releases the internal work guard, then blocks the calling - thread until all outstanding work tracked by + Blocks the calling thread until all outstanding work + tracked by @ref executor_type::on_work_started and @ref executor_type::on_work_finished completes. After all work finishes, joins the worker threads. @@ -115,15 +136,16 @@ class BOOST_CAPY_DECL exit; `join()` still waits for all threads to be joined before returning. - This function is idempotent. The first call performs the - join; subsequent calls return immediately. + Only the first call performs the join. A later call returns + immediately, even while the first call is still blocking on + another thread. @pre Must not be called from a thread in this pool (undefined behavior). @par Postconditions - All worker threads have been joined. The pool cannot be - reused. + If this call performed the join, all worker threads have + been joined. The pool cannot be reused. @par Thread Safety May be called from any thread not in this pool. @@ -158,7 +180,7 @@ class BOOST_CAPY_DECL get_executor() const noexcept; }; -/** An executor that submits work to a thread_pool. +/** Runs continuations on the worker threads of a thread_pool. Executors are lightweight handles that can be copied and stored. All copies refer to the same underlying thread pool. @@ -228,16 +250,22 @@ class thread_pool::executor_type If the calling thread is a worker of this pool, returns `c.h` for symmetric transfer so the caller can resume the - continuation inline. Otherwise, posts the continuation to - the pool for execution on a worker thread and returns - `std::noop_coroutine()`. + continuation inline. Otherwise, posts the continuation as + @ref post does and returns `std::noop_coroutine()`. On that + path, the continuation is not run if the pool stops first, + and a call that races with @ref thread_pool::stop can go + either way. See @ref post. @param c The continuation to execute. On the post path, - must remain at a stable address until dequeued - and resumed. + must remain at a stable address until resumed or + abandoned. @return `c.h` when the calling thread is a pool worker; `std::noop_coroutine()` otherwise. + + @throws std::system_error If this call posts and must start + the pool's worker threads, and a thread cannot be started. + `c` is not queued. */ BOOST_CAPY_DECL std::coroutine_handle<> @@ -245,11 +273,26 @@ class thread_pool::executor_type /** Post a continuation to the thread pool. - The continuation is resumed on one of the pool's - worker threads. The continuation must remain at a stable - address until it is dequeued and resumed. + The continuation is resumed on one of the pool's worker + threads, unless the pool stops first. The pool stops when + @ref thread_pool::stop is called, or when + @ref thread_pool::join has finished waiting for outstanding + work. A continuation still queued when the pool stops, or + posted after it stops, is not run. It stays queued until + the pool is destroyed, which abandons it. + + A call that races with @ref thread_pool::stop on another + thread can therefore go either way, and nothing in the call + tells the caller which. + + The continuation must remain at a stable address until it + is resumed or abandoned. @param c The continuation to execute. + + @throws std::system_error If this call must start the pool's + worker threads, and a thread cannot be started. `c` is not + queued. */ BOOST_CAPY_DECL void diff --git a/test/doc/programs/quick_start_hello.cpp b/test/doc/programs/quick_start_hello.cpp index 0d5e943a4..ce9209dd1 100644 --- a/test/doc/programs/quick_start_hello.cpp +++ b/test/doc/programs/quick_start_hello.cpp @@ -1,5 +1,6 @@ // // Copyright (c) 2026 Steve Gerbino +// Copyright (c) 2026 Michael Vandeberg // // Distributed under the Boost Software License, Version 1.0. (See accompanying // file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) @@ -38,7 +39,7 @@ int main() capy::run_async(pool.get_executor())(greet()); // join() waits for outstanding work to complete; the pool - // destructor only stops the pool and discards pending work + // destructor does not wait, and abandons any work still queued pool.join(); } // end::full[] diff --git a/test/doc/reference/thread_pool.record.cpp b/test/doc/reference/thread_pool.record.cpp index 9490c3525..6a177d48d 100644 --- a/test/doc/reference/thread_pool.record.cpp +++ b/test/doc/reference/thread_pool.record.cpp @@ -40,7 +40,7 @@ void run_on_pool() auto ex = pool.get_executor(); capy::run_async(ex)(some_task()); // start work; tracked so join() waits for it pool.join(); // wait for outstanding work to complete - // pool destructor stops the pool, discarding any pending work + // pool destructor stops the pool and abandons any work still queued } // end::example[] } // namespace ex_1