Redox retry and email handling#117
Open
toddkazakov wants to merge 2 commits into
Open
Conversation
Redox order processing retried far more than the configured 3 attempts. The inner RetryingConsumer caps attempts, but on exhaustion it returned the error, so the offset was never committed and the fault-tolerant consumer restarted and redelivered the message - reprocessing in batches of 3 indefinitely (a poison-message loop). Add an opt-in CommitOnFailure option to RetryOptions that logs and swallows the final error once retries are exhausted, so the offset is committed and the message is not redelivered. Enable it for the redox consumers, which already use a 3-attempt RetryOptions. Thread the existing zap logger into RetryingConsumer so the give-up is logged with structured context.
GetEmailAddressFromOrder only ever read the first entry of the email list and returned a hard error when it failed to parse, which caused order processing to fail on a single invalid email. Iterate the full patient/guarantor email list and return the first address that parses, skipping invalid or non-string entries. When none are valid, proceed without an email instead of erroring.
toddkazakov
force-pushed
the
tk-redox-retry-and-email-handling
branch
from
June 4, 2026 16:03
3e91892 to
2a73213
Compare
ewollesen
reviewed
Jun 18, 2026
| ) | ||
| if err != nil && r.opts.CommitOnFailure { | ||
| // Give up and commit the offset so the message is not redelivered. | ||
| r.logger.Warnw("giving up on message and committing offset", "offset", cm.Offset, "attempts", r.opts.Attempts, zap.Error(err)) |
Contributor
There was a problem hiding this comment.
I think it would make sense if we monitored the rate of these errors and alerted if they were elevated beyond some threshold.
ewollesen
approved these changes
Jun 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.