Report await sites for tasks parked on a pending future

[sarsko]

Stacked on #352 — review/merge that one first. This PR is based on lazy-deadlock-backtraces; retarget it to main once that lands.

A backtrace fundamentally cannot find where an async task is stuck after the fact: when a future returns Poll::Pending its poll stack unwinds, and the await chain lives on in the compiler-generated state machine, which no unwinder can walk.

Shuttle worked around this by capturing eagerly inside its own leaf futures (Acquire::poll, JoinHandle::poll). That covers Shuttle primitives and nothing else — a user future that returns Pending on its own terms reported nothing but the spawn or block_on call site.

How

The hook with the right timing is the waker. A future returning Pending is contractually obliged to arrange a wakeup, and the ordinary way is cx.waker().clone() — which runs inside the future's own poll, on the live stack, through a vtable we own. Capture there and it works for arbitrary user futures.

New await_backtrace module:

• note_waker_clone() — called from the waker vtable's clone
• take_captured() — called by the driver loops once poll has returned Pending
• PollGuard — marks the extent of a driver-loop poll, so clones made by the executor itself (e.g. Task::waker()) are not mistaken for await sites
• InternalBlockOnGuard — marks the block_on that BatchSemaphore::acquire_blocking uses to implement the synchronous Mutex/RwLock. A task parked there keeps its whole call chain on its coroutine stack, so it is captured lazily on deadlock instead. This is the hot path, and skipping it is what keeps the flag cheap.

This subsumes and removes the special-case capture in Acquire::poll — that site clones the waker, so the generic hook already covers it.

take_captured() returning None when no clone happened is deliberate: a future may skip the clone when it already holds an equivalent waker, and falling back to the lazy path is better than printing a stale backtrace from an earlier, unrelated park.

Results

Deadlock reports now name the exact user frame, with line numbers, for all four shapes:

blocking pattern	reported
synchronous primitive	user_fn_locks
spawned future on a Shuttle primitive	user_sem_inner, user_sem_outer
block_on in a thread task	user_sem_inner, user_sem_outer
arbitrary user future, no Shuttle primitive	MyFuture::poll, user_arb_inner, user_arb_outer

Example — a hand-written future awaiting something never produced (Shuttle-internal frames trimmed for readability):

deadlock! blocked tasks: [main-thread (task main-thread(0), pending future)
 0: <Receiver as core::future::future::Future>::poll   at example.rs:120
 1: wait_for_config_update::{{closure}}                 at example.rs:126   // rx.await
 2: service_startup::{{closure}}                        at example.rs:130   // wait_for_config_update(rx).await
 3: shuttle_std::future::block_on                       at future.rs:354
 4: example_b_never_sent::{{closure}}                   at example.rs:139

Before this change, that task reported only frame 4.

Cost

4-thread mutex workload; 5 samples, same machine and run:

	SHUTTLE_CAPTURE_BACKTRACE=1	flag unset
main	2.565 s	29.6 ms
after #352	828 ms	29.1 ms
after this PR	30.7 ms	30.1 ms

So synchronous blocking now pays essentially nothing for the flag.

Async debug runs get more expensive, and I want that explicit: the hook fires on every waker clone within a poll rather than once per Pending. With the flag set, the future:: test module goes from 362 s to 626 s (16.6 s with the flag unset).

I chose fidelity over speed — keeping the last clone's stack is correct under select!/join!, where several children clone in one poll. Capturing only the first clone per poll would roughly halve the cost but pick an arbitrary branch. Happy to flip that if reviewers prefer.

With the flag unset there is no regression. The second commit on this branch makes that true: the first version left PollGuard and take_captured outside the flag check, which cost ~1-4% on the default async path. counter async benches with the flag unset, against unmodified main:

bench	before gating	after gating
pct-narrow	−2.1% (ns)	−3.51% (p=0.00)
random-narrow	+0.80% (p=0.00)	+1.53% (ns)
pct-wide	−0.56% (ns)	−2.08% (p=0.00)
random-wide	+3.70% (p=0.01)	−1.18% (p=0.00)

Three of four now come out slightly ahead of main, because gating also skips the ExecutionState::with that main performs unconditionally on every Pending poll in JoinHandle::poll, plus the Acquire::poll site this series removes outright.

Testing

• cargo fmt --check, cargo clippy --workspace --all-targets, cargo doc all clean
• cargo test --release -p shuttle: 426 passed, 0 failed
• All other workspace crates pass
• With the flag set: the four-pattern matrix above, plus future:: (81) and basic:: (356)

Known limitation

A future that returns Pending without cloning the waker (skipping it via a will_wake check when it already holds an equivalent one) is still invisible. It falls back to the lazy path, which for a future-polling task yields the bare poll loop.

Follow-up, not in this PR

format_for_deadlock uses {:#?}, which selects std's Debug: one long { fn: .., file: .., line: .. } record per frame, wrapped in Some(, with absolute paths. Switching to {} (std's Display) gives the numbered at file:line form that RUST_BACKTRACE=1 prints and shrinks the message ~7%. Independent of the capture mechanism, so I left it out.

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.