Summary
The akka.persistence.r2dbc.query.buffer-size setting controls how many events are buffered in memory during projection replay, but it is purely count-based (default: 1000). When individual events are large (e.g. several hundred KB each), a buffer of 1000 events can consume hundreds of MB or more, causing OOM errors.
Problem
When a view projection does a full replay over a large number of events with large payloads, the count-based buffer can exhaust available heap. The workaround is to manually reduce buffer-size, but this requires knowing the average event size ahead of time and is fragile — the right value differs per service and per deployment.
Proposed solution
Add an optional byte-size cap to the query buffer so that the buffer stops filling before it exceeds a configured memory threshold, regardless of event count:
akka.persistence.r2dbc.query {
buffer-size = 1000 # existing: max number of events
buffer-max-bytes = 64 MiB # new: optional max total payload bytes in buffer
}
When buffer-max-bytes is set, the buffer would stop accepting new entries once the cumulative serialized payload size exceeds the limit, even if the count has not yet reached buffer-size. This provides a memory-sensitive upper bound that automatically adapts to event sizes without per-service tuning.
Context
- Projection buffer config lives in
reference.conf under akka.persistence.r2dbc.query.buffer-size
- Event replay uses this same buffer
- The same pattern (large-event OOM on projection replay) has been observed across multiple independent deployments
Summary
The
akka.persistence.r2dbc.query.buffer-sizesetting controls how many events are buffered in memory during projection replay, but it is purely count-based (default: 1000). When individual events are large (e.g. several hundred KB each), a buffer of 1000 events can consume hundreds of MB or more, causing OOM errors.Problem
When a view projection does a full replay over a large number of events with large payloads, the count-based buffer can exhaust available heap. The workaround is to manually reduce
buffer-size, but this requires knowing the average event size ahead of time and is fragile — the right value differs per service and per deployment.Proposed solution
Add an optional byte-size cap to the query buffer so that the buffer stops filling before it exceeds a configured memory threshold, regardless of event count:
When
buffer-max-bytesis set, the buffer would stop accepting new entries once the cumulative serialized payload size exceeds the limit, even if the count has not yet reachedbuffer-size. This provides a memory-sensitive upper bound that automatically adapts to event sizes without per-service tuning.Context
reference.confunderakka.persistence.r2dbc.query.buffer-size