Skip to content

Add byte-size limit to projection query buffer to prevent OOM on large events #741

Description

@JustinPihony

Summary

The akka.persistence.r2dbc.query.buffer-size setting controls how many events are buffered in memory during projection replay, but it is purely count-based (default: 1000). When individual events are large (e.g. several hundred KB each), a buffer of 1000 events can consume hundreds of MB or more, causing OOM errors.

Problem

When a view projection does a full replay over a large number of events with large payloads, the count-based buffer can exhaust available heap. The workaround is to manually reduce buffer-size, but this requires knowing the average event size ahead of time and is fragile — the right value differs per service and per deployment.

Proposed solution

Add an optional byte-size cap to the query buffer so that the buffer stops filling before it exceeds a configured memory threshold, regardless of event count:

akka.persistence.r2dbc.query {
  buffer-size = 1000          # existing: max number of events
  buffer-max-bytes = 64 MiB   # new: optional max total payload bytes in buffer
}

When buffer-max-bytes is set, the buffer would stop accepting new entries once the cumulative serialized payload size exceeds the limit, even if the count has not yet reached buffer-size. This provides a memory-sensitive upper bound that automatically adapts to event sizes without per-service tuning.

Context

  • Projection buffer config lives in reference.conf under akka.persistence.r2dbc.query.buffer-size
  • Event replay uses this same buffer
  • The same pattern (large-event OOM on projection replay) has been observed across multiple independent deployments

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions