Remove buffer_memory param unsupported in kafka-python 3.x#174
Conversation
|
Warning Review limit reached
More reviews will be available in 9 minutes and 51 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe Kafka CLI no longer defines ChangesKafka configuration update
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Hello @hyp3rion31gr3y, but they also suggested an alternative to use. https://kafka-python.readthedocs.io/en/stable/upgrade_to_3_0.html#buffer-memory-removed docs says Please check if this is concerning else ignore :) Thanks |
hyp3rion31gr3y
force-pushed
the
fix-remove-buffer-memory
branch
from
bea99b1 to
e7d4021
Compare
The buffer_memory producer config was removed in kafka-python 3.0, causing ValueError on pod startup. Replace it with max_request_size (per-request bound) as recommended by the kafka-python 3.0 upgrade docs, and add the corresponding --kafka-max-request-size CLI argument. According to Claude(for later reference): ● These are fundamentally different parameters, so they shouldn't be set equal: - buffer_memory (33554432 = 32 MB) — Total memory pool the producer could use to buffer all unsent records across all partitions. It was an overall memory budget. - max_request_size (1048576 = 1 MB) — Maximum size of a single request (effectively the max record size). It's a per-request cap, not a memory pool. They control different things, so copying the old 32 MB value into max_request_size would be incorrect — it would allow 32 MB individual requests, which is almost certainly not what you want and could hit broker-side message.max.bytes limits. The default of 1048576 (1 MB) is kafka-python's own default for max_request_size and aligns with the typical Kafka broker default for message.max.bytes. I'd keep it as-is unless you have records larger than 1 MB that need to go through. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
hyp3rion31gr3y
force-pushed
the
fix-remove-buffer-memory
branch
from
e7d4021 to
02c80b3
Compare
|
lgtm |
|
Great, thank you! Also thanks Vishal for cross-checking with release notes. |
3 participants
The buffer_memory producer config was removed in kafka-python 3.0, causing ValueError on pod startup. Drop the parameter from the producer init and the corresponding --kafka-buffer-memory CLI argument.