Skip to content

馃挜 Enforce payload and memo size limits - #3149

Draft
jmaeagle99 wants to merge 1 commit into
mainfrom
payload-limits-enforcement
Draft

jmaeagle99 wants to merge 1 commit into
mainfrom
payload-limits-enforcement

Conversation

@jmaeagle99

Copy link
Copy Markdown
Contributor

What was changed

Workers now enforce the namespace's payload and memo size limits before sending, and every client warns about oversized payloads.

Validation at the gRPC layer. A new interceptor in temporal-serviceclient checks every WorkflowService request on the final message, after user interceptors have run. Fields over a warning threshold are logged with [TMPRL1103] and still sent. When a call carries error limits and a field exceeds them, the call fails locally with INVALID_ARGUMENT and is never sent or retried.

Worker enforcement. Workers read the namespace's blob and memo error limits from the DescribeNamespace call they already make at startup. They attach the limits only to their own completion and heartbeat calls, so other client calls only warn. Each worker turns a rejected call into a retryable failure instead of sending it:

  • An oversized workflow task completion fails the task with cause PAYLOADS_TOO_LARGE.
  • An oversized activity result or cancellation fails the activity with an ApplicationFailure of type PayloadsTooLarge.
  • An oversized heartbeat fails the activity and cancels it, as the server would. The activity's own later result is then not sent.

Each case records a PayloadsTooLarge failure reason on the existing task failure metrics.

Configuration.

  • WorkflowServiceStubsOptions.Builder.setPayloadLimits sets the warning thresholds: 512 KiB per payload and 2 KiB per memo by default, with 0 disabling a warning. It is experimental.
  • WorkerOptions.Builder.setDisablePayloadErrorLimit turns off worker enforcement.

Why?

The server already rejects oversized payloads, but harshly: it terminates the workflow, or fails the activity non-retryably. Failing the task retryably in the worker lets a corrected workflow or activity be redeployed and recover. This also brings Java in line with the core-based SDKs and Go, which already enforce these limits.

Checklist

  1. Closes SDK should fail workflow task if payloads size it known to be too large - Java聽#2833
  2. Closes Warn if the SDK tried to send a payload above a specific size - Java聽#3059
  3. How was this tested:
    • Unit tests cover the interceptor and options against an in-process gRPC server, including requests changed by user interceptors. They also cover each worker failure path with mocked stubs.
    • Integration tests against the dev server cover workflow task, activity result and heartbeat enforcement, client calls that are only warned, and the opt-out.
  4. Any docs updates needed?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Warn if the SDK tried to send a payload above a specific size - Java SDK should fail workflow task if payloads size it known to be too large - Java

1 participant