Skip to content

feat!(extstore): add new concurrency controls for external storage. - #3114

Open
cconstable wants to merge 5 commits into
mainfrom
extstore-concurrency-controls
Open

cconstable wants to merge 5 commits into
mainfrom
extstore-concurrency-controls

Conversation

@cconstable

@cconstable cconstable commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

What was changed

ExternalStorage.Builder.setConcurrency takes two limits:

  • maxDriverOperations caps requests in flight for a single client or worker. Each client and each worker gets its own limit.
  • maxOperationsPerMessage caps requests for a single message (workflow task activation, etc). Each message gets its own budget.

Both limits are cooperative. Drivers receive a limiter on the store and retrieve contexts and must wrap each request in permit. Requests made outside a permit are not counted, and a warning is logged when a driver completes without taking one.

context.getLimiter().permit(payload, () -> {
    // store/retrieve logic
});

Removed ExternalStorage.Builder.setMaxConcurrentPayloadVisits. The payload visitor now walks a message's payload sites maxOperationsPerMessage at a time. The bundled S3 driver takes permits.

Why?

The previous control bounded payload visits rather than driver calls, so setting it to 3 did not limit external storage requests to 3. It also only applied to workflow tasks, leaving client, activity, and Nexus paths unbounded.

maxDriverOperations is scoped to a client or worker so that ExternalStorage stays configuration. Holding the counter there meant two clients built from one config silently shared a limit.

Checklist

Added tests

@cconstable
cconstable requested a review from a team as a code owner October 2, 2026 00:46
@cconstable
cconstable force-pushed the extstore-concurrency-controls branch from d9f2581 to 6177558 Compare October 14, 2026 19:14

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant