Skip to content

v0.5.0: accepted cross-request work can be discarded during event retirement #207

Description

@Poggen

Related to #200, which is listed as fixed in v0.5.0. We observed a possible remaining edge case on that release, commit 12d5b6333fe52717325addcfe1e99e9fd4f77bcd.

A Durable Object workflow intermittently stops progressing during asynchronous work spanning requests. Two unmodified source-built controls timed out before the application’s build operation was admitted.

A logging-only diagnostic build captured this sequence:

  1. Context A hands operation 1788 to context B; B adopts it.
  2. B hands operation 1790, with Handler lifetime, back to A. A’s mailbox accepts it.
  3. A retires with one operation still queued.
  4. Cleanup removes operation 1790’s promise resolver. No adoption or delivery of that operation is recorded.

Expected: normal event retirement should not discard accepted referenced work.

Suspected cause: InFlight::finished() checks adopted operations, while handed operations are drained later in the wake path. The driver can therefore decide to finish while its mailbox contains referenced work. There also appears to be a window between the completion check and mailbox closure.

We tried a local correction that makes mailbox closure atomic with the retirement decision and adopts queued operations before the cross-entry wait helper checks completion. Five focused native tests pass, and the unchanged application recovery scenario passes in two fresh runs.

The existing negative control specifically removes the wait-helper adoption change: four tests pass, while the wrapper regression fails with:

retirement recheck bypassed queued operation adoption

That negative control tests part of the proposed correction; it is not a test-only reproduction against untouched v0.5.0.

Limitations: we have not reduced this to a standalone Worker reproduction or identified operation 1790’s precise application purpose. The trace establishes that accepted work was discarded; connecting that operation to the application stall remains an inference supported by the corrected runs.

This investigation and candidate patch were AI-assisted. We have retained the diagnostic trace, focused tests and patch, and can provide them if useful. Does this look like a remaining case of #200, or a separate event-retirement issue?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions