Derive unique periods from UTC scheduled time - #1377
Merged
Merged
Conversation
`UniqueOpts.ByPeriod` keys are computed before an insert's effective `ScheduledAt` is applied, so a job scheduled for tomorrow is keyed by today's period. Re-enqueueing "tomorrow at 9am" at different times of day therefore inserts duplicates, and the key disagrees with the period the job actually runs in. The period bound is also formatted in the time's own location. A process whose local time zone isn't UTC, or a caller passing a `ScheduledAt` in another zone, embeds an offset like `-05:00` in the key, so two processes in different zones compute different keys for the same period and both insert. Compute the unique key after the effective scheduled time is known, and normalize the period bound to UTC before formatting it. Truncation already operates on absolute time, so only the textual form changes. Document both behaviors on `ByPeriod` and add a changelog entry that notes the one-time key change for scheduled jobs and non-UTC processes during a rolling upgrade. Cover non-UTC clocks, non-UTC `ScheduledAt`, and zone-independent period boundaries in key construction; scheduled jobs deduplicating by scheduled period across insertion times while unscheduled and next-period jobs don't conflict; and inserts in different zones deduplicating end to end across every driver.
bgentry
force-pushed
the
bg/unique-by-period-utc
branch
from
September 24, 2026 13:36
0416292 to
4ff1b65
Compare
brandur
approved these changes
Sep 24, 2026
`JobGetAvailable_LargeMetadata` uses the same nanosecond timestamp for insertion and fetch. PostgreSQL stores timestamps at microsecond precision, so rounding can place the scheduled time after the fetch cutoff and make the benchmark intermittently fetch no job. Truncate the benchmark timestamp to microseconds before insertion so the stored scheduled time matches the fetch cutoff exactly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
UniqueOpts.ByPeriodpromises "at most one of these jobs per period", for example one daily digest per user per day. Two bugs could break that promise and let duplicates through.Scheduled jobs were bucketed by when they were inserted, not when they run. Say you insert a daily digest at 10am today with
ScheduledAtset to 9am tomorrow, and then the same digest is inserted again at 8am tomorrow, also for 9am tomorrow. Both jobs are meant to run at the same moment, but the first was keyed to today's period and the second to tomorrow's, so River saw them as different and ran the digest twice.The period's timezone leaked into the unique key. The start of the period was written into the key using the local timezone of whichever process inserted the job. A server running in
America/Chicagowrote the start of a day-long period as2026-09-23T19:00:00-05:00, while a server running in UTC wrote the same instant as2026-09-24T00:00:00Z. The text differed, so the keys differed, and both servers could insert the "unique" job. The same happened when a caller passed aScheduledAtin a non-UTC zone.This change computes the unique key only after the job's effective scheduled time is known, so a scheduled job is deduplicated against other jobs scheduled in the same period. It also always writes the period in UTC. Periods were already calculated on absolute time, so which jobs fall into which period is unchanged; only the way the period is written into the key is fixed.
Because unique keys change for scheduled
ByPeriodjobs and for anyByPeriodjob inserted from a process whose timezone isn't UTC, an old and a new client running side by side during a rolling upgrade may each insert one copy of such a job for the same period. Jobs that aren't scheduled and are inserted from UTC processes keep exactly the same keys. The changelog calls this out.